Commit 5c79209d authored by Paul Gierz's avatar Paul Gierz

update docs; some small stuff for scope

parent 33d3fb35
...@@ -100,7 +100,7 @@ todo_include_todos = False ...@@ -100,7 +100,7 @@ todo_include_todos = False
# The theme to use for HTML and HTML Help pages. See the documentation for # The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes. # a list of builtin themes.
# #
html_theme = "alabaster" html_theme = "sphinx_rtd_theme"
# Theme options are theme-specific and customize the look and feel of a # Theme options are theme-specific and customize the look and feel of a
# theme further. For a list of options available for each theme, see the # theme further. For a list of options available for each theme, see the
......
SCOPE Documentation SCOPE Documentation
=================== ===================
Welcome to the ``scope`` documentation. ``scope`` is a two-in-one command line
utility as well as a Python library designed to facilitate coupling and
communication between various Earth system models. The minimal quickstart::
$ pip install scope-coupler
$ scope --help
$ scope preprocess ${CONFIG_FILE} echam
$ scope regrid ${CONFIG_FILE} pism
The commands listed above would install the scope coupler; show you what it can
do, gather relevant files from the atmosphere model ``echam``, and regrid them
onto a ``pism`` ice sheet grid.
However, ``scope`` is capable of much more than this. You can preprocess or
postprocess data on either side of the communication, modify variable names and
attributes, perform corrections due to resolution differences, and provide your
own specific steps for each part of the coupling process.
``scope`` is designed to run completely independently of the models being used,
the run-time infrastructure available on the supercomputer, and, perhaps most
importantly **requires 0 modifications to your model code**. To get started,
have a look at the documentation below.
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2
:caption: Contents: :caption: Contents:
readme
installation installation
usage usage
modules API
contributing contributing
authors authors
history history
......
...@@ -8,31 +8,31 @@ scope.cli module ...@@ -8,31 +8,31 @@ scope.cli module
---------------- ----------------
.. automodule:: scope.cli .. automodule:: scope.cli
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
scope.models module scope.models module
------------------- -------------------
.. automodule:: scope.models .. automodule:: scope.models
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
scope.scope module scope.scope module
------------------ ------------------
.. automodule:: scope.scope .. automodule:: scope.scope
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
Module contents Module contents
--------------- ---------------
.. automodule:: scope .. automodule:: scope
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
============= ===========
Library Usage Using SCOPE
============= ===========
An example configuration file is provided under ``examples/scope_config.yaml``: ``scope`` uses configuration files, generally written in the ``YAML`` syntax,
to define what you wantt it do. This configuration is divided into seperate
sections. Here, we give a brief overview. A complete reference is currently
being written.
The first section, simply titled ``scope``, defines general parameters for the program:
.. code-block:: yaml
:linenos:
scope:
couple_dir: "/path/to/directory/"
number openMP processes: 8
In the example above, we define two things for the program.
``couple_dir``
Line 2
This entry defines where scope will save it's files. These files
generally include remap weights to be reused each time the coupler is
called; gathered output files for processing for the other model; and
intermediate files that may be interesting for diagnosis.
``number openMP processes``
Line 3
Since ``scope`` uses ``cdo`` in the background; you can use this to
define how many processes you want to run ``cdo`` on. This generally
speeds up regridding.
Next, there is a section which may optionally be defined,
``template_replacements``. Here, you can store key/value pairs which will be
replaced elsewhere in the configuration. As an example:
.. code-block:: yaml
:linenos:
template_replacements:
EXP_ID: "PI_1x10"
DATE_PATTERN: "[0-9]{6}"
Now, any other time that ``{{ EXP_ID }}`` is used in the configuration, it will
be replaced with ``PI_1x10``. The syntax here is derived from the Jinja2 Python
package used for templating.
.. warning::
I'm not sure what happens here if you try to use recursion in the template
replacements!
You can also see in this section that you can define a ``DATE_PATTERN``.
Specific key/value pairs ending with the substring ``PATTERN`` are treated as a
regular expression.
Next, you describe the models you wish to couple together.
.. code-block:: yaml
:linenos:
model_name:
type: physical_domain
griddes: built-in CDO grid description, or path to a SCRIP formatted file
outdata_dir: /some/path
code table: build-in CDO code table, or path to a file with GRB-style code table
send:
...send directives...
receive:
...receive directives...
pre_step:
...description of pre step...
post_step:
...description of post step...
In the generalised example above, we define:
``model_name``
A model to couple, in this case, ``model_name``. Usually, this would be
more specific, e.g. ``echam``, ``openifs``, ``pism``, ``fesom``.
Inside the ``model_name`` configuration, we again have:
``type``
This describes the *type* of the model; e.g. atmosphere, ice, ocean.
``griddes``
Here, you must specify which grid description to use for this model.
This is the default for this model.
``outdata_dir``
This defines where scope will, by default, look for files for this
particular model. However, you can override this on a case by case
basis. See the send directives for more information.
``code_table``
Since ``scope`` is built on top of ``cdo``, which supports ``grb``
files, here you can specify which code table to use in order to detect
variable names when converting from ``grb`` to ``netcdf``.
``send``
This configuration contains send directives for other coupling partners.
More on this in the next section.
``receive``
This configuration is used to receive information from other models.
``pre_step``
Programs run before a particular step. Can be configured for each step
separately, e.g. ``pre_preprocess``, or ``pre_regrid``.
``post_step``
Programs run after a particular step
Example: Configuration Files for ``SCOPE``
------------------------------------------
A complete example configuration file is provided under
``examples/scope_config.yaml``:
.. literalinclude:: ../examples/scope_config.yaml .. literalinclude:: ../examples/scope_config.yaml
:language: yaml :language: yaml
:linenos: :linenos:
This section describes how to use ``scope`` from a Python program.
Example: ``PISM`` to ``ECHAM6``
-------------------------------
Command line interface
----------------------
``scope`` comes with a command line interface. For a very quick introduction::
$ scope --help
This will print usage information.
Python Library Usage
--------------------
While the command line interface is nice for users who never want to actually
touch ``scope`` code; we also support the ability to use scope functions in
your own ``Python`` programs. this section describes how to use ``scope`` from
a script.
To use scope in a project:: To use scope in a project::
import scope import scope
...good luck from this point forward. Paul still needs to write docs. Consider
having a look at the developer API.
...@@ -17,6 +17,13 @@ pism: ...@@ -17,6 +17,13 @@ pism:
interp: con interp: con
transformation: transformation:
- expr: "dBdt=...." - expr: "dBdt=...."
scripts:
- script1
- script1_args:
-script1_flags:
infile: fjdsflksa
outfile: jfkls
send: send:
solid_earth: solid_earth:
thk: thk:
......
# @Author: Paul Gierz <pgierz>
# @Date: 2019-12-05T07:26:48+01:00
# @Email: pgierz@awi.de
# @Filename: scope.py
# @Last modified by: pgierz
# @Last modified time: 2020-02-27T16:38:38+01:00
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Here, the ``scope`` library is described. This allows you to use specific parts Here, the ``scope`` library is described. This allows you to use specific parts
...@@ -484,10 +492,12 @@ class Preprocess(Scope): ...@@ -484,10 +492,12 @@ class Preprocess(Scope):
code_table = var_dict.get( code_table = var_dict.get(
"code table", self.config[self.whos_turn].get("code table") "code table", self.config[self.whos_turn].get("code table")
) )
code_table_command = f"-t {code_table}" if code_table else ""
cdo_command = ( cdo_command = (
self.get_cdo_prefix() self.get_cdo_prefix()
+ " -f nc -t " + " -f nc "
+ code_table + " "
+ code_table_command
+ " -select,name=" + " -select,name="
+ varname + varname
+ " " + " "
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment