Workflow

The workflow is what the user would like to achieve through lemmings, be it a simple chained run, a chained run with mesh refinement or whatever floats your boat.

A Workflow configures the scheme your run will follow, from a simple recursion, to a mesh adaptation, a post-processing operation, or customized operations. A Workflow is set up by two files :

• {workflow_name}.py : Python script of the scheme.

• {workflow_name}.yml : Yaml registering specific properties of your run’ Workflow.

NOTE: the workflow configuration file (.yml) and script (.py) MUST have the same prefix for the moment!!!

The configuration {workflow_name}.yml

Every workflow requires a configuration file. This file will be completed by the end user of lemmings. The number of parameters/informations required is workflow dependent, so make sure you know them or contact your super-user of lemmings.

#---Required Settings--#
exec: |                      #your environement, slurm exec, module etc (writted in the batch)
      echo "hello world exec"
      python twicer.py
exec_pj: |                  # exec for the PJ (writted in the batch)
      echo "hello world exec_pj"
job_queue: long         # name of the job queue based on your '{machine}.yml'
pjob_queue: short        # name of the post-job queue based on your '{machine}.yml'

There are 4 mandatory parameters that have to be present in each configuration file:

• exec : Here, write what you want to put in your Batch file instead of the “-EXEC-” string from the {machine}.yml.

• exec_pj: the same as exec but for the post-job queue. You may just need to source your virtual environment to execute lemmings post-job

• job_queue/pjob_queue: The name of the queue you want to use for the Job/Post-job of lemmings. Queues are listed in the {machine}.yml configuration file.

The script {workflow_name}.py

The workflow is defined through a class called LemmingJob and sets the framework that must be followed by the user. The structure is represented below

          +--------------+ True   +-----------+                     +------------+True +-------------+
Start----->Check on Start+------->|Prepare Run+--->Job submission--->Check on end+----->After End Job+------>Happy
          +-----+--------+    ^   +-----------+                     +------+-----+     +-------------+        End
                |             |                                            |
                |False        |                                            |False
                |             |                   +-------------+          |
                              |                   |  Prior to   |          |
              Early           +-------------------+new iteration<----------+
               End                                +-------------+

For an actual example of a workflow.py definition, please refer to the tutorials.