4. Run the Medium-Range Weather (MRW) Application

Attention

This section assumes that the user has already built the MRW App. If this is not the case, see Chapter 3 for instructions on how to build the App.

The Unified Forecast System (UFS) Medium-Range Weather (MRW) Application can be run in coupled or uncoupled mode. Chapter 4.2 describes how to run a forecast in uncoupled mode (atmosphere-only), and Chapter 4.3 describes how to run a forecast in coupled mode.

The steps required for generating an experiment are as follows:

Download and stage data

Generate the workflow experiment

Run the workflow

Optional: Plot the output

4.1. Download and Stage the Data

The MRW App requires input files to run. These include static datasets, initial and boundary conditions files, and model configuration files. On Level 1 and 2 systems, the data required to run MRW App tests are already available. For Level 3 and 4 systems, the data must be added. Detailed instructions on how to add the data can be found in Section 7.4 Downloading and Staging Input Data. Sections 7.1 and 7.2 contain useful background information on the input and output files used in the MRW App.

4.2. Running in Uncoupled Mode

4.3. Running in Coupled Mode

4.4. Generate the Forecast Experiment

Create a COMROT and EXPDIR. The experiment and workflow set-up scripts in following steps will point to these paths. Initial conditions will also need to be placed in COMROT.

# Make these dirs in directory above mrw dir

mkdir COMROT EXPTDIR (for uncoupled)
mkdir COMROT EXPTDIR ICSDIR HOMEDIR (for coupled - HOMEDIR is optional)

HOMEDIR holds files generated by expt (archives) EXPTDIR holds all config files created by expt, plus crontab & xml files –> can modify config.base COMROT –> must put ICs into this directory for uncoupled. For coupled, no. COMROT also contains outputs and other stuff during the runs. (non-scrubbed output)

(generate components (from setup_expts.py, then generate experiments)

Go to to ush/rocoto dir.

Run ./setup_expt.py to set up the expt. - No DA, so do forecast-only argument. - --pslot - YYYYMMDDHH_test (expt name) - idate and edate must be the same for forecast only expts (rather than DA)–> even tho it starts from start end day. Should correspond to start date. - --app argument is optional. (Check setup_expt.py for the options.) - –resdet 384 –> what resolution to run at? For coupled, only 384 is supported. For uncoupled all res < 768 are supported. - cold start is default. warm start is if you have data from past experiment.

For “free-forecast” aka uncoupled/atmoshere only: ./setup_expt.py forecast-only –pslot 2013100100_test –idate 2013100100 –edate 2013100100 –app ATM –resdet 384 –start cold –comrot /work/noaa/epic-ps/ufs-mrw-v2.0/coupled/COMROT –expdir /work/noaa/epic-ps/ufs-mrw-v2.0/coupled/EXPDIR/C384

(assumes you built with S2SWA default build).

For coupled: coupled exp setup: ./setup_expt.py forecast-only –pslot 2013100100_test –idate 2013100100 –edate 2013100100 –app S2SW –resdet 384 –start cold –comrot /work/noaa/epic-ps/ufs-mrw-v2.0/coupled/COMROT –expdir /work/noaa/epic-ps/ufs-mrw-v2.0/coupled/EXPDIR/C384 –icsdir /work/noaa/epic-ps/ufs-mrw-v2.0/coupled/ICSDIR

A run creates RUNDIRS which contains scrubbed output files.

4.4.1. For uncoupled:

Edit config.base vi config.base

Change: ACCOUNT, user-specific paths (HOMEDIR, STMP, PTMP, NOSCRUB) Experiment-specific environment parameters: SDATE (start), EDATE (end–>list same as start!!!), (no idea what FSTART is. don’t mess.)

Run experiment generator script:

cd ufs-mrweather-app/global-workflow/ush/rocoto
[on Orion] module load contrib/0.1; module load rocoto/1.3.3
[on Hera] module use -a /contrib/anaconda/modulefiles
module load anaconda/anaconda3-5.3.1
./setup_expt.py forecast-only --pslot $EXP_NAME --idate YYYYMMDDCC --edate YYYYMMDDCC--resdet desired_resolution --gfs_cyc 4 --comrot $PATH_TO_YOUR_COMROT_DIR --expdir $PATH_TO_YOUR_EXPDIR

Example with COMROT and EXPDIR paths:

./setup_expt.py forecast-only --pslot test --idate 2020010100 --edate 2020010118 --resdet 384 --gfs_cyc 4 --comrot /work/noaa/stmp/cbook/COMROT --expdir /work/noaa/epic-ps/cbook/uncoupled/EXPDIR

This will generate $PSLOT (specific experiment name) folders in COMROT and EXPDIR, with config files in $EXPDIR/$PSLOT

Copy IC files into COMROT/$PSLOT. Directory name should be like: gfs.YYYYMMDDCC, with structure: gfs.$YYYYMMDD/CC/atmos. INPUT folder within …/atmos/ contains sfc files needed for GFS ATM to run. Edit config.base in $EXPDIR/$PSLOT (ACCOUNT, HOMEDIR, STMP/PTMP, HPSSARCH)

Run ./setup_workflow_fcstonly.py --expdir $EXPDIR/$PSLOT. This will generate crontab and .xml files for the experiment in $EXPDIR/$PSLOT.

4.4.2. Description of Workflow Tasks

Note

This section gives a general overview of workflow tasks. To begin running the workflow, skip to Step 4.5.1

Individual tasks that make up the workflow are specified in the FV3LAM_wflow.xml file. Table 4.1 describes the function of each baseline task. The first three pre-processing tasks; MAKE_GRID, MAKE_OROG, and MAKE_SFC_CLIMO are optional. If the user stages pre-generated grid, orography, and surface climatology fix files, these three tasks can be skipped by adding the following lines to the config.sh file before running the generate_FV3LAM_wflow.sh script:

RUN_TASK_MAKE_GRID="FALSE"
RUN_TASK_MAKE_OROG="FALSE"
RUN_TASK_MAKE_SFC_CLIMO="FALSE"

The setup_expt.py file runs the specific j-job scripts (global-workflow/jobs/J[task_name]) in the prescribed order when the experiment is launched via setup_workflow.py or the rocotorun command. Each j-job task has its own source script (or “ex-script”) named ex[task_name].sh in the global-workflow/scripts directory. Two database files named FV3LAM_wflow.db and FV3LAM_wflow_lock.db are generated and updated by the Rocoto calls. There is usually no need for users to modify these files. To relaunch the workflow from scratch, delete these two *.db files and then call the launch script repeatedly for each task.

Table 4.1 Baseline workflow tasks in the SRW App
Workflow Task	Task Description


run_fcst	Run the forecast model (UFS weather model)
run_post	Run the post-processing tool (UPP)

In addition to the baseline tasks described in Table 4.1 above, users may choose to run some or all of the METplus verification tasks. These tasks are described in Table 4.2 below.

Table 4.2 Verification (VX) workflow tasks in the SRW App
Workflow Task	Task Description
GET_OBS_CCPA	Retrieves and organizes hourly CCPA data from NOAA HPSS. Can only be run if `RUN_TASK_GET_OBS_CCPA="TRUE"` and user has access to NOAA HPSS data.
GET_OBS_NDAS	Retrieves and organizes hourly NDAS data from NOAA HPSS. Can only be run if `RUN_TASK_GET_OBS_NDAS="TRUE"` and user has access to NOAA HPSS data.
GET_OBS_MRMS	Retrieves and organizes hourly MRMS composite reflectivity and echo top data from NOAA HPSS. Can only be run if `RUN_TASK_GET_OBS_MRMS="TRUE"` and user has access to NOAA HPSS data.
VX_GRIDSTAT	Runs METplus grid-to-grid verification for 1-h accumulated precipitation
VX_GRIDSTAT_REFC	Runs METplus grid-to-grid verification for composite reflectivity
VX_GRIDSTAT_RETOP	Runs METplus grid-to-grid verification for echo top
VX_GRIDSTAT_##h	Runs METplus grid-to-grid verification for 3-h, 6-h, and 24-h (i.e., daily) accumulated precipitation. Valid values of `##` are `03`, `06`, and `24`.
VX_POINTSTAT	Runs METplus grid-to-point verification for surface and upper-air variables
VX_ENSGRID	Runs METplus grid-to-grid ensemble verification for 1-h accumulated precipitation. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_REFC	Runs METplus grid-to-grid ensemble verification for composite reflectivity. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID = "TRUE"`.
VX_ENSGRID_RETOP	Runs METplus grid-to-grid ensemble verification for echo top. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_##h	Runs METplus grid-to-grid ensemble verification for 3-h, 6-h, and 24-h (i.e., daily) accumulated precipitation. Valid values of `##` are `03`, `06`, and `24`. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_MEAN	Runs METplus grid-to-grid verification for ensemble mean 1-h accumulated precipitation. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_PROB	Runs METplus grid-to-grid verification for 1-h accumulated precipitation probabilistic output. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_MEAN_##h	Runs METplus grid-to-grid verification for ensemble mean 3-h, 6-h, and 24h (i.e., daily) accumulated precipitation. Valid values of `##` are `03`, `06`, and `24`. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_PROB_##h	Runs METplus grid-to-grid verification for 3-h, 6-h, and 24h (i.e., daily) accumulated precipitation probabilistic output. Valid values of `##` are `03`, `06`, and `24`. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_PROB_REFC	Runs METplus grid-to-grid verification for ensemble probabilities for composite reflectivity. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSGRID_PROB_RETOP	Runs METplus grid-to-grid verification for ensemble probabilities for echo top. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSGRID="TRUE"`.
VX_ENSPOINT	Runs METplus grid-to-point ensemble verification for surface and upper-air variables. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSPOINT="TRUE"`.
VX_ENSPOINT_MEAN	Runs METplus grid-to-point verification for ensemble mean surface and upper-air variables. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSPOINT="TRUE"`.
VX_ENSPOINT_PROB	Runs METplus grid-to-point verification for ensemble probabilities for surface and upper-air variables. Can only be run if `DO_ENSEMBLE="TRUE"` and `RUN_TASK_VX_ENSPOINT="TRUE"`.

4.5. Run the Workflow

4.5.1. Run Using Rocoto

Submit job through crontab by copying entry in $PSLOT.crontab into crontab via crontab -e.

Monitor status of workflow using rocotostat:

rocotostat -d /path/to/workflow/database/file -w /path/to/workflow/xml/file [-c YYYYMMDDCCmm,[YYYYMMDDCCmm,...]] [-t taskname,[taskname,...]] [-s] [-T]

For example: .. code-block:: console

rocotostat -d $PSLOT.db -w $PSLOT.xml

Check status of specific task/job:

rocotocheck -d </path/to/workflow/database/file> -w </path/to/workflow/xml/file> -c YYYYMMDDCCmm -t taskname

4.5.2. Run Manually (Without Rocoto)

4.6. Plot the Output

Two python scripts are provided to generate plots from the post-processed GRIB2 output. Information on how to generate the graphics can be found in Chapter 8.