Warning, /doc/old_doc/devel_HOWTO.sgml is written in an unsupported language. File is not indexed.
view on githubraw file Latest commit ad38444b on 2018-01-31 20:35:48 UTC
8eab949e1c Ed H*0001 <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
072a8eef5d Jean*0002 <!--
0003 -->
8eab949e1c Ed H*0004
0005 <article id="MITgcm-Development-HOWTO">
0006
0007 <articleinfo>
0008 <title>MITgcm Development HOWTO</title>
0009
0010 <author>
0011 <firstname>Ed</firstname>
0012 <surname>Hill III</surname>
0013 <affiliation>
0014 <address><email>eh3@mit.edu</email></address>
0015 </affiliation>
0016 </author>
0017
0018 <revhistory>
0019 <revision>
0020 <revnumber>0.01</revnumber>
fdb5287ce5 Ed H*0021 <date>2003-08-07</date>
8eab949e1c Ed H*0022 <authorinitials>eh3</authorinitials>
0023 <revremark>
0024 Initial version.
0025 </revremark>
0026 </revision>
880d91dbcc Jean*0027 <revision>
0028 <revnumber>0.02</revnumber>
0029 <date>2010-01-21</date>
0030 <authorinitials>jmc</authorinitials>
0031 <revremark>
f7a66e0ebc Jean*0032 Update links.
880d91dbcc Jean*0033 </revremark>
0034 </revision>
74aa04f48b Jean*0035 <revision>
0036 <revnumber>0.03</revnumber>
0037 <date>2010-04-25</date>
0038 <authorinitials>jmc</authorinitials>
0039 <revremark>
0040 Add subsection "Developer settings" (under CVS Repository).
0041 </revremark>
0042 </revision>
f7a66e0ebc Jean*0043 <revision>
0044 <revnumber>0.04</revnumber>
0045 <date>2011-04-24</date>
0046 <authorinitials>jmc</authorinitials>
0047 <revremark>
0048 Update subsection "The verification suite".
0049 </revremark>
0050 </revision>
8eab949e1c Ed H*0051 </revhistory>
0052
0053 <abstract>
0054 <para>This document describes how to develop software for the
0055 MITgcm project.</para>
0056 </abstract>
0057 </articleinfo>
0058
0059 <sect1 id="intro">
0060 <title>Introduction</title> <para>The purpose of this document is
0061 to help new developers get "up to speed" with MITgcm
0062 development.</para>
0063 <sect2>
0064 <title>New Versions of This Document</title> <para>You can
0065 obtain the latest version of this document <ulink
880d91dbcc Jean*0066 url="http://mitgcm.org/public/docs.html">online</ulink> in
8eab949e1c Ed H*0067 various formats.</para>
0068 </sect2>
0069 <sect2>
0070 <title>Feedback and corrections</title> <para>If you have
0071 questions or comments about this document, please feel free to
0072 <ulink url="mailto:MITgcm-support@mitgcm.org">contact the
0073 authors</ulink>.
0074 </para>
0075 </sect2>
0076 </sect1>
0077
0078 <sect1 id="background">
0079 <title>Background</title>
0080
0081 <sect2>
0082 <title>User Manual</title>
0083
5643c8ebe0 Ed H*0084 <para>Before jumping into development, please familiarize yourself with
880d91dbcc Jean*0085 the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual
5643c8ebe0 Ed H*0086 </ulink>. This document contains volumes of useful information and is
0087 included here by reference.</para>
8eab949e1c Ed H*0088
5643c8ebe0 Ed H*0089 <!--
0090 <para>Also, a "snapshot" or <ulink
8eab949e1c Ed H*0091 url="http://mitgcm.org/dev_docs/">development version</ulink> of
0092 the user manual may be available, though this is only put on the
0093 web for testing purposes.</para>
5643c8ebe0 Ed H*0094 -->
8eab949e1c Ed H*0095 </sect2>
0096
0097 <sect2>
0098 <title>Prerequisites</title> <para>To develop for MITgcm project
0099 you will need a UNIX or UNIX-like set of build tools including
0100 the following:</para>
0101 <blockquote>
0102 <simplelist type="inline">
0103 <member>CVS client</member>
0104 <member>make or (preferably) GNU make</member>
0105 <member>FORTRAN compiler</member>
0106 <member>C compiler</member>
0107 <member>[ba]sh and [t]csh shells</member>
0108 <member>PERL</member>
0109 <member>LaTeX and LaTeX2HTML</member>
0110 </simplelist>
0111 </blockquote>
0112 <para>Essentially all of the work described here has been tested
0113 on recent versions of Red Hat Linux (eg. 7.3 through 9). Except
0114 where noted, all shell commands will be provided using bash
0115 syntax.
0116 </para>
0117 </sect2>
0118
0119 </sect1>
0120
0121 <sect1 id="cvs">
0122 <title>CVS Repository</title>
74aa04f48b Jean*0123
8eab949e1c Ed H*0124 <sect2>
0125 <title>Layout</title>
0126
0127 <para>Unlike many open source projects, the MITgcm CVS tree does
0128 not follow a simple "src", "docs", "share", and "test" directory
0129 layout. Instead, there are multiple higher-level directories
0130 that each, to some extent, depend upon the presence of the
0131 others. The tree currently resembles:</para>
0132
0133 <programlisting>gcmpack/
74aa04f48b Jean*0134 CVSROOT -hidden-
0135
0136 MITgcm code
0137 bin empty
0138 doc basic developpment documentation
0139 eesupp execution environment support code (wrapper)
0140 exe empty
f7a66e0ebc Jean*0141 jobs runtime shell scripts for
74aa04f48b Jean*0142 various platforms (not maintained)
0143 lsopt line search
0144 model main dynamics (core)
0145 optim line search interface
0146 pkg alternate and optional numerics, etc.
0147 tools scripts to build (and test)
0148 utils pre/post processing tools (matlab, ..)
0149 verification standard regression tests + examples
0150 + documented examples (tutorials)
0151 tutorial_examples (only in release1 branch)
0152
0153 MITgcm_contrib contributed code
0154
0155 acesgrid.org build acesgrid web site
8eab949e1c Ed H*0156 development experimental stuff
74aa04f48b Jean*0157 gfd_lab -?-
072a8eef5d Jean*0158 manual source of MITgcm documentation
8eab949e1c Ed H*0159 mitgcm.org build web site
f7a66e0ebc Jean*0160 old_develop old and early development source
072a8eef5d Jean*0161 misc -?-
0162 models -?-
0163 packages -?-
0164 preprocess -?-
74aa04f48b Jean*0165 pdfs some pdfs
0166 planetinabottle.org unfinished web site
0167 www.ecco-group.org build ecco web site ?
8eab949e1c Ed H*0168 </programlisting>
0169
74aa04f48b Jean*0170 <!--
8eab949e1c Ed H*0171 <para>Efforts are underway to reduce the complexity.</para>
74aa04f48b Jean*0172 -->
8eab949e1c Ed H*0173
0174 </sect2>
0175
0176 <!--
0177 <sect2>
0178 <title>Releases</title> <para>Currently, there are two main
0179 branches:</para>
0180 <itemizedlist mark="bullet">
0181 <listitem>
0182 <para>Development</para>
0183 <itemizedlist mark="bullet">
0184 <listitem>
0185 <para>MAIN</para>
0186 </listitem>
0187 <listitem>
0188 <para>ecco-branch</para>
0189 </listitem>
0190 </itemizedlist>
0191 </listitem>
0192 <listitem>
0193 <para>Production</para>
0194 <itemizedlist mark="bullet">
0195 <listitem>
0196 <para>Release1</para>
0197 </listitem>
0198 <listitem>
0199 <para>Release2</para>
0200 </listitem>
0201 </itemizedlist>
0202 </listitem>
0203 </itemizedlist>
0204 </sect2>
0205 -->
0206
0207 <sect2>
0208 <title>Branches</title>
0209
0210 <para>As shown in the online <ulink
74aa04f48b Jean*0211 url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph">
b6ae7becbd Dimi*0212 ViewCVS-generated tree</ulink>, the MITgcm codebase is split into
0213 branches or "lines" under which development proceeds. The main line
0214 of development is referred to as the "MAIN" version of the code.
8eab949e1c Ed H*0215 </para>
0216
0217 <para>Periodically, a "Release" branch is formed from the "MAIN"
fc48f16730 Ed H*0218 development branch. This is done in order to create a relatively stable
0219 reference point for both users and developers. The intent is that once a
b6ae7becbd Dimi*0220 release branch has been created, only bug-fixes will be added to it.
fc48f16730 Ed H*0221 Meanwhile, development (which might "break" or otherwise render invalid
0222 the documentation, tutorials, and/or examples contained within a release
b6ae7becbd Dimi*0223 branch) is allowed to continue along the MAIN line.</para>
8eab949e1c Ed H*0224 </sect2>
0225
0226 <sect2>
74aa04f48b Jean*0227 <title> Developer settings </title>
0228
f7a66e0ebc Jean*0229 <para>CVS is a convenient tool to keep up-to-date a personal copy of the
0230 MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html">
0231 using CVS </ulink>). The same tool is used by developers to
74aa04f48b Jean*0232 incorporate any change into the repository. However, this later
8ba4d46446 Jean*0233 function requires specific settings, as detailed here after</para>
74aa04f48b Jean*0234 <orderedlist>
0235 <listitem>
8ba4d46446 Jean*0236 <para> You will need an account (login access) to the server
6e06ec5d94 Jean*0237 "mitgcm.org" (curently: <filename>mitgcmcvs.mit.edu</filename>)
0238 with the proper group setting (e.g., group "gcmctrb" to add/modify
8ba4d46446 Jean*0239 code into MITgcm_contrib).
6e06ec5d94 Jean*0240 This kind of account is granted only upon well motivated request
0241 (we recommend to ask your senior MITgcm-collaborator to send such
0242 request to marshall-admin at techsquare.com with Cc to Chris Hill
8ba4d46446 Jean*0243 for approval).</para>
6e06ec5d94 Jean*0244 <para> The access to the server <filename>mitgcm.org</filename> is
0245 through ssh-key authorization which will need to be set properly on
0246 both side (on your local machine and on your server account).
0247 You need to be able to ssh to <filename>mitgcm.org</filename>
8ba4d46446 Jean*0248 (or <filename>ssh MY_USER_NAME@mitgcm.org</filename>
0249 in case of different user-name on both sides) to proceed further.</para>
74aa04f48b Jean*0250 </listitem>
0251
0252 <listitem>
f7a66e0ebc Jean*0253 <para> You need to register to the
77d43ff20a Jean*0254 <ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-cvs">
8ba4d46446 Jean*0255 mitgcm-cvs </ulink> mailing list.
f7a66e0ebc Jean*0256 This ensures that other developers will receive email notification
8ba4d46446 Jean*0257 when you make changes; you will also receive such email
74aa04f48b Jean*0258 when others make changes to the repository.
0259 </para>
0260 </listitem>
0261
0262 <listitem>
f7a66e0ebc Jean*0263 <para> It is highly recommended that you register also to the
77d43ff20a Jean*0264 <ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel">
8ba4d46446 Jean*0265 mitgcm-devel </ulink> mailing list (expect a short delay for
74aa04f48b Jean*0266 this request to be processed).
0267 This list is intended for developer discussions.
0268 </para>
0269 </listitem>
0270
0271 <listitem>
8ba4d46446 Jean*0272 <para> The standard CVS-anonymous mode (using "cvsanon",
0273 as mentionned <ulink url="http://mitgcm.org/public/source_code.html">
0274 here </ulink>) does not provide check-in ("cvs commit") permission.
74aa04f48b Jean*0275 Instead, you will need to set our CVS environment as follow:</para>
0276 <screen>
0277 $ export CVS_RSH=ssh
0278 $ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack'
0279 </screen>
8ba4d46446 Jean*0280 <para> The reason for such limitation is that when downloading a directory,
0281 e.g.: <filename>myCopy</filename>, from the CVS repository (e.g.,
74aa04f48b Jean*0282 <filename>MITgcm_contrib/thisPart</filename>) using the command:</para>
0283 <screen>
0284 $ cvs co -P -d myCopy MITgcm_contrib/thisPart
0285 </screen>
f7a66e0ebc Jean*0286 <para> the type of CVS environment which has been used
8ba4d46446 Jean*0287 is stored in the file <filename>myCopy/CVS/Root</filename>.
0288 This prevent to re-use, for cvs-commit purpose,
f7a66e0ebc Jean*0289 a cvs local copy (<filename>myCopy</filename>) which was obtained
74aa04f48b Jean*0290 using the CVS anonymous mode.</para>
0291 </listitem>
0292
0293 <listitem>
f7a66e0ebc Jean*0294 <para> At this stage, you should be able to send your modified source
0295 file (e.g., <filename>src_file</filename>) from your local copy directory
74aa04f48b Jean*0296 (<filename>myCopy</filename>) to the CVS repository
0297 (<filename>MITgcm_contrib/thisPart</filename>) using the command
0298 "cvs commit":</para>
0299 <screen>
0300 $ cd myCopy
0301 $ cvs -n update (optional; check if new changes have been made)
0302 $ cvs diff src_file (optional; list your changes)
0303 $ cvs commit src_file
0304 </screen>
f7a66e0ebc Jean*0305 <para> It is essential that you provide a short description of the
0306 changes you made to <filename>src_file</filename> as you check-in
0307 this file (the "cvs commit" command automatically opens your standard
74aa04f48b Jean*0308 editor for this purpose).</para>
8ba4d46446 Jean*0309 <para> Note:
6e06ec5d94 Jean*0310 Please ignore the following warnings that the "cvs commit" command
8ba4d46446 Jean*0311 produces if you are not part of the "gcmpack" group:
0312 <screen>
0313 cvs commit: failed to create lock directory for `/u/gcmpack/CVSROOT'
0314 (/u/gcmpack/CVSROOT/#cvs.history.lock): Permission denied
0315 cvs commit: failed to obtain history lock in repository `/u/gcmpack'
0316 </screen>
0317 These warnings are not affecting the changes you made to the CVS repository.
0318 </para>
74aa04f48b Jean*0319 </listitem>
0320
0321 </orderedlist>
0322
0323 </sect2>
0324
0325 <sect2>
0326 <title>Main code development</title>
5dc09eb5d9 Jean*0327 <para><emphasis>(formerly named "Tagging" ; this section needs an update)
0328 </emphasis></para>
8eab949e1c Ed H*0329
fc48f16730 Ed H*0330 <para>The intent of tagging is to create "known-good" checkpoints that
0331 developers can use as references. Traditionally, MITgcm tagging has
0332 maintained the following conventions:</para>
8eab949e1c Ed H*0333
0334 <orderedlist>
0335 <listitem>
fc48f16730 Ed H*0336 <para>Developer checks out code into a local CVS-managed directory,
0337 makes various changes/additions, tests these edits, and eventually
0338 reaches a point where (s)he is satisfied that the changes form a new
0339 "useful" point in the evolution of the code.</para>
8eab949e1c Ed H*0340 </listitem>
0341
0342 <listitem>
0343 <para>The developer then runs the <ulink
880d91dbcc Jean*0344 url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport">
0345 testreport</ulink> shell script to see if any problems are introduced.
fc48f16730 Ed H*0346 While not intended to be exhaustive, the test cases within the
0347 verification directory do provide some indication whether gross errors
0348 have been introduced.
8eab949e1c Ed H*0349 </para>
0350 </listitem>
0351
0352 <listitem>
0353 <para>Having satisfied him- or herself that the changes are
0354 ready to be committed to the CVS repository, the developer
0355 then:</para>
0356 <orderedlist>
0357 <listitem>
fc48f16730 Ed H*0358 <para>adds a "checkpointXY_pre" comment (where X is a checkpoint
0359 number and Y is a letter) to the <ulink
880d91dbcc Jean*0360 url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index">
fc48f16730 Ed H*0361 tag-index</ulink> file and checks it into the CVS
0362 repository</para>
8eab949e1c Ed H*0363 </listitem>
0364 <listitem>
fc48f16730 Ed H*0365 <para>submits the set of changes to the CVS repository and adds
0366 comments to <filename>tag-index</filename> describing what the
0367 changes are along with a matching "checkpointXY_post" entry</para>
8eab949e1c Ed H*0368 </listitem>
0369 </orderedlist>
0370 </listitem>
0371 </orderedlist>
0372
fc48f16730 Ed H*0373 <para>The result of this tagging procedure is a sequence of development
0374 checkpoints with comments which resembles:</para>
8eab949e1c Ed H*0375
0376 <programlisting>
0377 checkpoint50e_post
0378 o make KPP work with PTRACERS
0379 - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
0380 nearly a copy of kpp_transport_s
f7a66e0ebc Jean*0381 - there is no analogue to SurfaceTendencyS, so I have to use
8eab949e1c Ed H*0382 gPtr(of the surface layer) instead
0383 o add a new platform SunFire+mpi (SunFire 15000) to genmake
0384 checkpoint50e_pre
0385
0386 checkpoint50d_post
f7a66e0ebc Jean*0387 o change kpp output from multiple-record state files to single-record state
8eab949e1c Ed H*0388 files analogous to write_state.F
f7a66e0ebc Jean*0389 o reduce the output frequency of cg3d-related stuff to the monitor frequency,
0390 analogous to the cg2d-related output.
0391 o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
8eab949e1c Ed H*0392 so that writing to internal file fn (with length 512) fails.
0393 checkpoint50d_pre
0394 </programlisting>
0395
fc48f16730 Ed H*0396 <para>This information can be used to refer to various stages of the code
0397 development. For example, bugs can be traced to individual sets of CVS
0398 checkins based upon their first appearance when comparing the results from
0399 different checkpoints.</para>
8eab949e1c Ed H*0400
0401 </sect2>
0402 </sect1>
0403
fdb5287ce5 Ed H*0404 <sect1 id="coding">
9442059cdd Ed H*0405 <title>Coding for MITgcm</title>
0406
0407 <sect2 id="build_tools">
0408 <title>Build Tools</title>
0409
23e640c5e4 Ed H*0410 <para>Many Open Source projects use the "GNU Autotools" to help streamline
0411 the build process for various Unix and Unix-like architectures. For a
0412 user, the result is the common "configure" (that is,
0413 "<filename>./configure && make && make install</filename>") commands.
0414 For MITgcm, the process is similar. Typical commands are:</para>
9442059cdd Ed H*0415
0416 <screen>
5dc09eb5d9 Jean*0417 $ genmake2 -mods=../code
5643c8ebe0 Ed H*0418 $ make depend
0419 $ make
9442059cdd Ed H*0420 </screen>
0421
23e640c5e4 Ed H*0422 <para>The following sections describe the individual steps in the build
0423 process.</para>
f7a66e0ebc Jean*0424
9442059cdd Ed H*0425 <sect3 id="genmake">
0426 <title>The <filename>genmake2</> Utility</title>
0427
5dc09eb5d9 Jean*0428 <para><emphasis>(Note: the older <filename>genmake</>
0429 has been replaced by <filename>genmake2</>)</emphasis></para>
9442059cdd Ed H*0430
23e640c5e4 Ed H*0431 <para>The first step in any MITgcm build is to create a Unix-style
0432 <filename>Makefile</filename> which will be parsed by
0433 <filename>make</filename> to specify how to compile the MITgcm source
0434 files. For more detailed descriptions of what the make tools are and
0435 how they are used, please see:</para>
9442059cdd Ed H*0436
0437 <itemizedlist>
0438 <listitem>
23e640c5e4 Ed H*0439 <para><ulink url="http://www.gnu.org/software/make/make.html">
0440 http://www.gnu.org/software/make/make.html</></para>
9442059cdd Ed H*0441 </listitem>
0442 <listitem>
23e640c5e4 Ed H*0443 <para><ulink url="http://www.oreilly.com/catalog/make2/">
0444 http://www.oreilly.com/catalog/make2/</></para>
9442059cdd Ed H*0445 </listitem>
0446 </itemizedlist>
0447
23e640c5e4 Ed H*0448 <para>Genmake can often be invoked successfully with a command line as
0449 simple as:</para>
0450
0451 <screen>
5643c8ebe0 Ed H*0452 $ genmake2 -mods=../code
23e640c5e4 Ed H*0453 </screen>
0454
0455 <para>However, some systems (particularly commercial Unixes that lack a
0456 more modern "/bin/sh" implementation or that have shells installed in
0457 odd locations) may require an explicit shell invocation such as one of
0458 the following: </para>
0459
0460 <screen>
5643c8ebe0 Ed H*0461 $ /usr/bin/sh genmake2 -make=gmake -mods=../code
0462 $ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code
23e640c5e4 Ed H*0463 </screen>
0464
0465 <para>The genmake2 code has been written in a Bourne and BASH (v1)
0466 compatible syntax so it should work with most "sh" and all recent "bash"
0467 implementations.</para>
9442059cdd Ed H*0468
23e640c5e4 Ed H*0469 <para>As the name implies, <filename>genmake2</filename> generates a
0470 <filename>Makefile</filename>. It does so by first parsing the
0471 information supplied from the following sources</para>
9442059cdd Ed H*0472
0473 <orderedlist>
0474 <listitem>
bbda629aeb Jean*0475 <para>a <filename>gemake_local</filename> file in the current
23e640c5e4 Ed H*0476 directory</para>
9442059cdd Ed H*0477 </listitem>
0478 <listitem>
0479 <para>directly from command-line options</para>
0480 </listitem>
0481 <listitem>
23e640c5e4 Ed H*0482 <para>an "options file" as specified by the command-line option
0483 <filename>-optfile='FILENAME'</filename></para>
9442059cdd Ed H*0484 </listitem>
5dc09eb5d9 Jean*0485 <listitem>
0486 <para> a <filename>packages.conf</filename> file (in the current
0487 directory or in one of the "MODS" directories, see below)
0488 which contains the specific list of packages to compile
0489 </para>
0490 </listitem>
9442059cdd Ed H*0491 </orderedlist>
0492
23e640c5e4 Ed H*0493 <para>then checking certain dependency rules (the package dependencies),
0494 and finally writing a <filename>Makefile</filename> based upon the
0495 source code that it finds. For convenience within various Unix
0496 shells, <filename>genmake2</> supports both "long"- and "short"-style
0497 options. A complete list of the available options can be obtained
0498 from:</para>
9442059cdd Ed H*0499
0500 <screen>
5643c8ebe0 Ed H*0501 $ genmake2 -help
9442059cdd Ed H*0502 </screen>
0503
23e640c5e4 Ed H*0504 <para>The most important options for <filename>genmake2</> are:</para>
9442059cdd Ed H*0505
0506 <variablelist>
0507
0508 <varlistentry>
0509 <term><filename>--optfile=/PATH/FILENAME</></term>
23e640c5e4 Ed H*0510
9442059cdd Ed H*0511 <listitem>
23e640c5e4 Ed H*0512 <para>This specifies the "options file" that should be used for a
0513 particular build. The options file is a convenient and
0514 machine-indepenent way of specifying parameters such as the
0515 FORTRAN compiler (<filename>FC=</>), FORTRAN compiler
0516 optimization flags (<filename>FFLAGS=</>), and the locations of
0517 various platform- and/or machine-specific tools
0518 (eg. <filename>MAKEDEPEND=</>). As with <filename>genmake2</>,
0519 all options files should be written to be compatible with
0520 Bourne--shell ("sh" or "BASH v1") syntax. Examples of various
0521 options files can be found in
0522 <filename>$ROOTDIR/tools/build_options</>.</para>
0523
0524 <para>If no "optfile" is specified (either through the command lin
0525 or the environment variable), genmake2 will try to make a
0526 reasonable guess from the list provided in
0527 <filename>$ROOTDIR/tools/build_options</>. The method used for
0528 making this guess is to first determine the combination of
0529 operating system and hardware (eg. "linux_ia32") and then find a
0530 working Fortran compiler within the user's path. When these
0531 three items have been identified, genmake2 will try to find an
0532 optfile that has a matching name. </para>
0533
0534 <para>Everyone is encouraged to submit their options files to the
0535 MITgcm project for inclusion (please send to
0536 <email>MITgcm-support@mitgcm.org</email>). We are particularly
0537 grateful for options files tested on new or unique
0538 platforms!</para>
9442059cdd Ed H*0539 </listitem>
23e640c5e4 Ed H*0540
9442059cdd Ed H*0541 </varlistentry>
0542
0543 <varlistentry>
5643c8ebe0 Ed H*0544 <term><filename>-adof=/path/to/file</></term>
0545 <term><filename>-adoptfile=/path/to/file</></term>
0546 <listitem>
0547 <para>This option specifies the "adjoint" or automatic
0548 differentiation options file to be used. The file is analogous
0549 to the "optfile" defined above but it specifies information for
0550 the AD build process. The default file is located in <filename>
0551 $ROOTDIR/tools/adjoint_options/adjoint_default </> and it
0552 defines the "TAF" and "TAMC" compilers. An alternate version is
0553 also available at <filename>
0554 $ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the
0555 newer "STAF" compiler. As with any compilers, it is helpful to
0556 have their directories listed in your $PATH environment
0557 variable.</para>
0558 </listitem>
0559 </varlistentry>
0560
0561 <varlistentry>
23e640c5e4 Ed H*0562 <term><filename>-mods=DIR</></term>
0563 <term><filename>-mods='DIR1 [DIR2 ...]'</></term>
0564 <listitem>
0565 <para>This option specifies a list of directories containing
0566 "modifications". These directories contain files with names
0567 that may (or may not) exist in the main MITgcm source tree but
0568 will be overridden by any identically-named sources within the
0569 "MODS" directories. The order of precedence for this
0570 "name-hiding" is as follows:</para>
0571 <itemizedlist>
0572 <listitem><para>"MODS" directories (in the order given)
0573 </para></listitem>
0574 <listitem><para>Packages either explicitly specified or
0575 provided by default (in the order given)</para></listitem>
0576 <listitem><para>Packages included due to package dependencies
f7a66e0ebc Jean*0577 (in the order that that package dependencies are
23e640c5e4 Ed H*0578 parsed)</para></listitem>
0579 <listitem><para>The "standard dirs" (which may have been
0580 specified by the "-standarddirs" option)</para></listitem>
0581 </itemizedlist>
5dc09eb5d9 Jean*0582 </listitem>
0583 </varlistentry>
0584
0585 <varlistentry>
0586 <term><filename>-pgroups=/PATH/FILENAME</></term>
0587 <listitem>
0588 <para>This option specifies the file where package groups are
0589 defined. If not set, the package-groups definition will
0590 be read from
0591 <filename>$ROOTDIR/pkg/pkg_groups</>.</para>
0592 <para>
0593 It also contains the default list of packages (defined
0594 as the group <filename>"default_pkg_list"</>) which is used
0595 when no specific package list (file: <filename>packages.conf</>)
0596 is found in current directory or in any "MODS" directory.
0597 </para>
0598 </listitem>
0599 </varlistentry>
23e640c5e4 Ed H*0600
5dc09eb5d9 Jean*0601 <varlistentry>
0602 <term><filename>-pdepend=/PATH/FILENAME</></term>
0603
0604 <listitem>
0605 <para>This specifies the dependency file used for packages. If
0606 not specified, the default dependency file is
0607 <filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for this file is
0608 parsed on a line-by-line basis where each line containes either a
0609 comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule
0610 where the "+" or "-" symbol specifies a "must be used with" or a
0611 "must not be used with" relationship, respectively. If no rule is
0612 specified, then it is assumed that the two packages are compatible
0613 and will function either with or without each other.</para>
23e640c5e4 Ed H*0614 </listitem>
0615 </varlistentry>
0616
0617 <varlistentry>
0618 <term><filename>-make=/path/to/gmake</></term>
9442059cdd Ed H*0619 <listitem>
23e640c5e4 Ed H*0620 <para>Due to the poor handling of soft-links and other bugs common
0621 with the <filename>make</> versions provided by commercial Unix
0622 vendors, GNU <filename>make</filename> (sometimes called
0623 <filename>gmake</filename>) should be preferred. This option
0624 provides a means for specifying the make program to be
0625 used.</para>
9442059cdd Ed H*0626 </listitem>
0627 </varlistentry>
0628
0629 </variablelist>
0630
23e640c5e4 Ed H*0631 <para>A successful run of <filename>genmake2</> will produce a
0632 <filename>Makefile</>, a <filename>PACKAGES_CONFIG.h</> file, and
0633 various convenience files used for the automatic differentiation
0634 process.</para>
0635
0636 <para>In general, it is best to use <filename>genmake2</> on a "clean"
0637 directory that is free of all source (*.[F,f],*.[F,f]90) and header
0638 (*.h,*.inc) files. Generally, this can be accomplished in an
5dc09eb5d9 Jean*0639 "un-clean" directory by running "make Clean" followed by "make
23e640c5e4 Ed H*0640 makefile".</para>
9442059cdd Ed H*0641
0642 </sect3>
0643
0644 <sect3 id="makefile_use">
5643c8ebe0 Ed H*0645 <title>Using the <filename>Makefile</></title>
9442059cdd Ed H*0646
5643c8ebe0 Ed H*0647 <para>Once a <filename>Makefile</> has been created using
0648 <filename>genmake2</>, one can build a "standard" (forward
0649 simulator) executable using:</para>
9442059cdd Ed H*0650
0651 <screen>
5dc09eb5d9 Jean*0652 $ make Clean
5643c8ebe0 Ed H*0653 $ make depend
0654 $ make
9442059cdd Ed H*0655 </screen>
0656
5dc09eb5d9 Jean*0657 <para>The "make Clean" step will remove any stale source files, include
23e640c5e4 Ed H*0658 files, and links. It is strongly recommended for "un-clean"
0659 directories which may contain the (perhaps partial) results of
5dc09eb5d9 Jean*0660 previous builds. Such "debris" can interfere with the next stage of
f7a66e0ebc Jean*0661 the build.
0662 A more agressive cleaning option, "make CLEAN", can be used to also
5dc09eb5d9 Jean*0663 remove the executable and output files from a previous run.</para>
23e640c5e4 Ed H*0664
0665 <para>The "make depend" step will create a large number of symbolic
0666 links from the local directory to the source file locations. It also
0667 parses these files and creates an extensive list of dependencies
0668 within the <filename>Makefile</> itself. The links that exist at this
0669 stage are mostly "large F" files (*.F and *.F90) that need to be
0670 processed by a C preprocessor ("CPP"). Since "make depend" edits the
0671 <filename>Makefile</>, it is important not to skip this step!</para>
0672
0673 <para>The final "make" invokes the C preprocessor to produce the "little
0674 f" files (*.f and *.f90) and then compiles them to object code using
0675 the specified FORTRAN compiler and options. An intermediate script is
0676 often used during this stage to further process (usually, make simple
0677 substitutions) custom definitions such as variable types within the
0678 source files. This additional stage is necessary in order to overcome
0679 some of the inconsistencies in the sizes of objects (bytes) between
5643c8ebe0 Ed H*0680 different compilers. The result of the build process is an executable
0681 with the name <filename>mitgcmuv</>.</para>
0682
0683 <para>In addition to the forward simulator described above, the
0684 <filename>Makefile</> also has a number of targets that can be used to
0685 produce various adjoint and tangent-linear builds for optimization and
0686 other parameter-sensitivity problems. The additional targets within
0687 the <filename>Makefile</> are:</para>
0688
0689 <variablelist>
0690
0691 <varlistentry>
0692 <term><filename>make adall</></term>
0693 <listitem>
0694 <para>This target produces an <filename>mitgcmuv_ad</> executable
0695 using the <filename>taf</> or <filename>staf</> adjoint
f7a66e0ebc Jean*0696 compiler. See the <filename>genmake2</> "-adof" option for
5643c8ebe0 Ed H*0697 compiler selection.</para>
0698 </listitem>
0699 </varlistentry>
0700
0701 <varlistentry>
0702 <term><filename>make ftlall</></term>
0703 <listitem>
0704 <para>Similar to <filename>make adall</> above, this
0705 produces...</para>
0706 </listitem>
0707 </varlistentry>
0708
0709 </variablelist>
9442059cdd Ed H*0710
5643c8ebe0 Ed H*0711 <para>Please report any compilation failures or other build problems to
0712 the <email>MITgcm-support@mitgcm.org</email> list.</para>
9442059cdd Ed H*0713
0714 </sect3>
0715
0716 </sect2>
0717
0718 <sect2 id="verification">
0719 <title>The Verification Suite</title>
0720
23e640c5e4 Ed H*0721 <para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>
19e7f79b1c Jean*0722 directory) includes many (> 90) examples intended for regression
f7a66e0ebc Jean*0723 testing. Each one of these test-experiment directories contains "known-good"
23e640c5e4 Ed H*0724 output files along with all the input (including both code and data
f7a66e0ebc Jean*0725 files) required for their re-calculation.
0726 Also included in <filename>$ROOTDIR/verification/</> is the shell
0727 script <filename>testreport</> to perform regression tests.</para>
0728
0729 <sect3 id="test-experiment">
0730 <title>Test-experiment Directory Content</title>
0731
0732 <para> Each test-experiment directory (<filename>TESTDIR</>) contains
0733 several standard subdirectories and files which <filename>testreport</>
0734 recognizes and uses when running a regression test.
0735 The directories/files that <filename>testreport</> uses are different
0736 for a forward test and an adjoint test (<filename>testreport -adm</>)
0737 and some test-experiments are set-up for only one type of regression
0738 test whereas others allow both types of tests (forward and adjoint).</para>
0739 <para>Also some test-experiment allows, using the same MITgcm executable,
0740 to perform multiple tests using different parameters and input files,
0741 with a primary input set-up
0742 (<filename>input/ </> or <filename>input_ad/ </>)
0743 and corresponding results (<filename>results/output.txt</> or
0744 <filename>results/output_adm.txt</>) and with one or several secondary
0745 inputs (<filename>input.OTHER/ </> or <filename>input_ad.OTHER/ </>)
0746 and corresponding results (<filename>results/output.OTHER.txt </>
0747 or <filename>results/output_adm.OTHER.txt</>).</para>
0748 <variablelist>
0749
0750 <varlistentry>
0751 <term>directory <filename>TESTDIR/results/</></term>
0752 <listitem>
0753 <para>contains reference standard output used for test comparison.
0754 <filename>results/output.txt</> and
0755 <filename>results/output_adm.txt</>
0756 correspond respectively to primary forward and adjoint test
0757 run on the reference platform (currently
0758 <filename>baudelaire.csail.mit.edu</>)
0759 on one processor (no MPI, single thread)
0760 using the reference compiler (curently the GNU fortran
0761 compiler <filename>gfortran</>).
0762 The presence of these files determines whether or not
0763 <filename>testreport</> is testing or skipping
0764 this test-experiment.
0765 Reference standard output for secondary tests
0766 (<filename>results/output.OTHER.txt</>
0767 or <filename>results/output_adm.OTHER.txt</>) are
0768 also expected here.</para>
0769 <para>The test comparison involves few model variables output, which are,
0770 by default and for a forward test, the 2-D solver initial residual
0771 (<filename>cg2d_init_res</>) and 3-D state variables (T,S,U,V)
0772 monitor output, and, by default and for an adjoint test, the
0773 cost-function and gradient-check. However, some test-experiments
0774 use some package-specific variable/monitor output according to
0775 the file <filename>TESTDIR/input[_ad][.OTHER]/tr_checklist</>
0776 specification.</para>
0777 </listitem>
0778 </varlistentry>
0779
0780 <varlistentry>
0781 <term>directory <filename>TESTDIR/build/</></term>
0782 <listitem>
0783 <para> initially empty directory where <filename>testreport</>
0784 will build the MITgcm executable for forward and adjoint test.
0785 It might contains an experiment specific
0786 <filename>genmake_local</> file (see <xref linkend="genmake">).
0787 </para>
0788 <para> Note that the original <filename>code[_ad]/SIZE.h_mpi</>
0789 is not directly used as "SIZE.h" to build an MPI-executable ;
0790 instead, a local copy <filename>build/SIZE.h.mpi</> is derived
0791 from <filename>code[_ad]/SIZE.h_mpi</>
0792 by adjusting the number of processors (nPx,nPy) according to
0793 <filename>NUMBER_OF_PROCS</> (see <xref linkend="testreport">,
0794 <filename>testreport -MPI</>) ; then it is linked to "SIZE.h"
0795 (<filename> ln -s SIZE.h.mpi SIZE.h </>) before building
0796 the MPI-executable.</para>
0797 </listitem>
0798 </varlistentry>
0799
0800 <varlistentry>
0801 <term>directory <filename>TESTDIR/code/</></term>
0802 <listitem>
0803 <para>contains the test-experiment specific source code
0804 used to build the MITgcm executable (<filename>mitgcmuv</>)
0805 for forward-test (using <filename>genmake2 -mods=../code</>).
0806 </para>
0807 <para> It can also contain specific source files with the suffix
0808 "<filename>_mpi</>" to be used
0809 <!--
0810 in the <filename>TESTDIR/build/</> directory
0811 -->
0812 in place of the corresponding file (without suffix)
0813 for an MPI test (see <xref linkend="testreport">).
0814 The presence or absence of <filename>SIZE.h_mpi</>
0815 determines whether or not an MPI test on this
0816 test-experiment is performed or skipped.</para>
0817 </listitem>
0818 </varlistentry>
0819
0820 <varlistentry>
0821 <term>directory <filename>TESTDIR/code_ad/</></term>
0822 <listitem>
0823 <para>contains the test-experiment specific source code
0824 used to build the MITgcm executable (<filename>mitgcmuv_ad</>)
0825 for adjoint-test (using <filename>genmake2 -mods=../code_ad</>).
0826 It can also contain specific source files with the suffix
0827 "<filename>_mpi</>" (see above).</para>
0828 </listitem>
0829 </varlistentry>
0830
0831 <varlistentry>
0832 <term>directory <filename>TESTDIR/input/</></term>
0833 <listitem>
0834 <para>contains the input and parameter files
0835 used to run the primary forward test of this test-experiment.
0836 </para>
0837 <para>It can also contain specific parameter files with the suffix
0838 "<filename>.mpi</>" to be used
0839 <!--
0840 in the <filename>TESTDIR/run/</> directory
0841 -->
0842 in place of the corresponding file (without suffix) for MPI
0843 test, or with suffix "<filename>.mth</>" to be used for
0844 multi-threaded test (see <xref linkend="testreport">).
0845 The presence or absence of <filename>eedata.mth</>
0846 determines whether or not a multi-threaded test on this
0847 test-experiment is performed or skipped.</para>
0848 <para>To save disk space and reduce downloading time, multiple
0849 copies of the same input file is avoided by using a shell
0850 script <filename>prepare_run</>.
0851 When such a script is found in <filename>TESTDIR/input/ </>,
0852 <filename>testreport</> run this script in directory
0853 <filename>TESTDIR/run/ </> after linking all the input file
0854 from <filename>TESTDIR/input/ </>.
0855 </para>
0856 </listitem>
0857 </varlistentry>
0858
0859 <varlistentry>
0860 <term>directory <filename>TESTDIR/input_ad/</></term>
0861 <listitem>
0862 <para>contains the input and parameter files
0863 used to run the primary adjoint test of this test-experiment.
0864 It can also contain specific parameter files with the suffix
0865 "<filename>.mpi</>" and shell script <filename>prepare_run</>
0866 as described above.</para>
0867 </listitem>
0868 </varlistentry>
0869
0870 <varlistentry>
0871 <term>directory <filename>TESTDIR/input.OTHER/</></term>
0872 <listitem>
0873 <para>contains the input and parameter files
0874 used to run the secondary OTHER forward test of this test-experiment.
0875 It can also contain specific parameter files with suffix
0876 "<filename>.mpi</>" or "<filename>.mth</>" and shell script
0877 <filename>prepare_run</> (see above).</para>
0878 <para> The presence or absence the file <filename>eedata.mth</>
0879 determines whether or not a secondary multi-threaded test on this
0880 test-experiment is performed or skipped.</para>
0881 </listitem>
0882 </varlistentry>
0883
0884 <varlistentry>
0885 <term>directory <filename>TESTDIR/input_ad.OTHER/</></term>
0886 <listitem>
0887 <para>contains the input and parameter files
0888 used to run the secondary OTHER adjoint test of this test-experiment.
0889 It can also contain specific parameter files with the suffix
0890 "<filename>.mpi</>" and shell script <filename>prepare_run</>
0891 (see above).</para>
0892 </listitem>
0893 </varlistentry>
0894 <!--
0895 -->
0896 <varlistentry>
0897 <term>directory <filename>TESTDIR/run/</></term>
0898 <listitem>
0899 <para> initially empty directory where <filename>testreport</>
0900 will run the MITgcm executable for primary forward and adjoint
0901 test.</para>
0902 <para>Symbolic links (using command "<filename>ln -s</>") are made
0903 for input and parameter files (from <filename>../input/ </>
0904 or from <filename>../input_ad/ </>) and for MITgcm executable
0905 (from <filename>../build/ </>) before the run proceeds.
0906 The sequence of links (function <filename>linkdata</> within shell
0907 script <filename>testreport</>) for a forward test is:</para>
0908 <screen>
0909 * link+rename or remove links
0910 to special files with suffix ".mpi" or ".mth" from ../input/
0911 * link files from ../input/
0912 * execute ../input/prepare_run (if it exists)
0913 </screen>
0914 <para>The sequence for an ajoint test is similar, with
0915 <filename>../input_ad/ </> replacing <filename>../input/ </>.
0916 </para>
0917 </listitem>
0918 </varlistentry>
0919
0920 <varlistentry>
0921 <term>directory <filename>TESTDIR/tr_run.OTHER/</></term>
0922 <listitem>
0923 <para> directory created by <filename>testreport</>
0924 to run the MITgcm executable for secondary "OTHER" forward
0925 or adjoint test.</para>
0926 <para> The sequence of links for a forward secondary test is:</para>
0927 <screen>
0928 * link+rename or remove links
0929 to special files with suffix ".mpi" or ".mth" from ../input.OTHER/
0930 * link files from ../input.OTHER/
0931 * execute ../input.OTHER/prepare_run (if it exists)
0932 * link files from ../input/
0933 * execute ../input/prepare_run (if it exists)
0934 </screen>
0935 <para>The sequence for an ajoint test is similar, with
0936 <filename>../input_ad.OTHER/ </> and <filename>../input_ad/ </>
0937 replacing <filename>../input.OTHER/ </> and <filename>../input/ </>.
0938 </para>
0939 </listitem>
0940 </varlistentry>
0941
0942 </variablelist>
0943 </sect3>
9442059cdd Ed H*0944
0945 <sect3 id="testreport">
0946 <title>The <filename>testreport</> Utility</title>
0947
f7a66e0ebc Jean*0948 <para> The shell script <filename>testreport</> (in
0949 <filename>$ROOTDIR/verification/</>), which was written to work with
0950 <filename>genmake2</>, can be used to build different versions
0951 of the MITgcm code, run the various examples, compare the output,
0952 and (if specified) email the results of each one of these tests to a
0953 central repository.</para>
23e640c5e4 Ed H*0954
0955 <para>On some systems, the testreport script can be run with a command
0956 line as simple as:</para>
0957
0958 <screen>
5643c8ebe0 Ed H*0959 $ cd verification
19e7f79b1c Jean*0960 $ ./testreport
23e640c5e4 Ed H*0961 </screen>
0962
0963 <para>However, some systems (those lacking or wiht a broken "/bin/sh")
0964 may require an explicit shell invocation such as:</para>
0965
0966 <screen>
19e7f79b1c Jean*0967 $ sh ./testreport -t 'exp2 exp4'
0968 $ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box'
23e640c5e4 Ed H*0969 </screen>
9442059cdd Ed H*0970
0971 <para>The <filename>testreport</> script accepts a number of
23e640c5e4 Ed H*0972 command-line options which can be listed using the <filename>-help</>
0973 option. The most important ones are:</para>
9442059cdd Ed H*0974
0975 <variablelist>
0976
0977 <varlistentry>
19e7f79b1c Jean*0978 <term><filename>-ieee</> (default) / <filename>-noieee</></term>
23e640c5e4 Ed H*0979 <listitem>
0980 <para>If allowed by the compiler (as defined in the "optfile"),
f7a66e0ebc Jean*0981 use IEEE arithmetic (<filename>genmake2 -ieee</>).
19e7f79b1c Jean*0982 This option, along with the gfortran / gcc compiler,
0983 is how the standard results are produced.</para>
0984 </listitem>
0985 </varlistentry>
0986
0987 <varlistentry>
0988 <term><filename>-optfile=/PATH/FILENAME</></term>
0989 <term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
0990 <listitem>
0991 <para>This specifies a list of "options files" that will be passed
0992 to <filename>genmake2</>. If multiple options files are used
0993 (say, to test different compilers or different sets of options
0994 for the same compiler), then each options file will be used with
0995 each of the test directories.</para>
23e640c5e4 Ed H*0996 </listitem>
0997 </varlistentry>
0998
0999 <varlistentry>
9442059cdd Ed H*1000 <term><filename>-tdir TESTDIR</></term>
1001 <term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term>
1002 <listitem>
23e640c5e4 Ed H*1003 <para>This option specifies the test directory or list of test
1004 directories that should be used. Each of these entries should
1005 exactly (note: they are case sensitive!) match the names of
19e7f79b1c Jean*1006 directories in <filename>$ROOTDIR/verification/</>. If this
23e640c5e4 Ed H*1007 option is omitted, then all directories that are properly
1008 formatted (that is, containing an <filename>input</>
1009 sub-directory and a <filename>results/output.txt</> file) will
1010 be used.</para>
9442059cdd Ed H*1011 </listitem>
1012 </varlistentry>
1013
1014 <varlistentry>
1015 <term><filename>-addr EMAIL</></term>
1016 <term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
1017 <listitem>
1018 <para>Send the results (namely, <filename>output.txt</>,
23e640c5e4 Ed H*1019 <filename>genmake_local</>, <filename>genmake_state</>, and
1020 <filename>Makefile</>) to the specified email addresses. The
1021 results are gzipped, placed in a tar file, MIME encoded, and
1022 sent to the specified address. If no email addresses are
1023 specified, no mail is sent.</para>
1024 </listitem>
1025 </varlistentry>
1026
1027 <varlistentry>
19e7f79b1c Jean*1028 <term><filename>-MPI NUMBER_OF_PROCS</></term>
23e640c5e4 Ed H*1029 <term><filename>-mpi</></term>
1030 <listitem>
f7a66e0ebc Jean*1031 <para>If the necessary file (<filename>TESTDIR/code/SIZE.h_mpi</>)
19e7f79b1c Jean*1032 exists, then use it (and all <filename>TESTDIR/code/*_mpi</> files)
f7a66e0ebc Jean*1033 for an MPI--enabled run. The new option (<filename>-MPI</> followed
1034 by the maximum number of processors) enable to build and run each
19e7f79b1c Jean*1035 test-experiment using variable number of MPI processors (multiple
f7a66e0ebc Jean*1036 of <filename>nPx*nPy</> from <filename>TESTDIR/code/SIZE.h_mpi</>
1037 and not larger than <filename>NUMBER_OF_PROCS</>).
1038 The short option ("-mpi") can only be used to build and run on 2 MPI
1039 processors (equivalent to "<filename>-MPI 2</>").</para>
19e7f79b1c Jean*1040 <para>Note that the use of MPI typically requires a
23e640c5e4 Ed H*1041 special command option (see "-command" below) to invoke the MPI
19e7f79b1c Jean*1042 executable. Examples of PBS scripts using testreport with MPI can be
23e640c5e4 Ed H*1043 found in the <ulink
19e7f79b1c Jean*1044 url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1045 tools/example_scripts directory</ulink>.</para>
23e640c5e4 Ed H*1046 </listitem>
1047 </varlistentry>
1048
1049 <varlistentry>
1050 <term><filename>-command='some command to run'</></term>
1051 <listitem>
19e7f79b1c Jean*1052 <para>For some tests, particularly MPI runs, a specific command
1053 might be needed to run the executable. This option allows a more general
23e640c5e4 Ed H*1054 command (or shell script) to be invoked. Examples of PBS scripts
19e7f79b1c Jean*1055 using testreport with MPI can be found in the <ulink
1056 url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1057 tools/example_scripts directory</ulink>.</para>
f7a66e0ebc Jean*1058 <para>
1059 For the case where the number of MPI processors varies according
1060 to each test-experiment, some key-words within the command-to-run
1061 argument will be replaced by their effective value:</para>
1062 <para>
1063 <filename>TR_NPROC </> will be replaced by the actual number
1064 of MPI processors needed to run the current test-experiment.</para>
1065 <para>
1066 <filename>TR_MFILE </> will be replaced by the name of local-file
1067 that testreport creates from the full list of machines which
1068 "<filename>testreport -mf MACHINE_FILE</>" provides, but truncated
1069 to the exact number of machines.</para>
1070 </listitem>
1071 </varlistentry>
1072
1073 <varlistentry>
1074 <term><filename>-mf MACHINE_FILE</></term>
1075 <listitem>
1076 <para>
1077 To use with <filename>-MPI NUMBER_OF_PROCS</> option, to specify
1078 the file containing the full list of <filename>NUMBER_OF_PROCS</>
1079 machines to use for the MPI runs.</para>
1080 </listitem>
1081 </varlistentry>
1082
1083 <varlistentry>
1084 <term><filename>-mth</></term>
1085 <listitem>
1086 <para>compile (with <filename>genmake2 -omp</>) and run with
1087 multiple threads (using eedata.mth).</para>
9442059cdd Ed H*1088 </listitem>
1089 </varlistentry>
1090
1091 </variablelist>
1092
f7a66e0ebc Jean*1093 <para>The <filename>testreport</> script will create an output directory
1094 <filename>tr_NAME_DATE_N/ </>, with <filename>hostname</> as default
1095 <filename>NAME</>, <filename>DATE</> the current date followed by
1096 a suffix number "N" to distinguish from previous
1097 <filename>testreport</> output directories.
1098 <filename>testreport</> writes progress to the screen (stdout)
1099 and reports into the ouput directory as it runs.
1100 In particular, one can find, in the ouput directory,
1101 the <filename>summary.txt</> file that contains a brief comparison
1102 of the current output with the "known-good" output.
1103 At the end of the testing process, the <filename>tr_out.txt</> file
1104 is generated in <filename>$ROOTDIR/verification/ </>
1105 as a compact version of <filename>summry.txt</> file.</para>
1106
1107 </sect3>
9442059cdd Ed H*1108
f7a66e0ebc Jean*1109 <sect3 id="tst_2plus2">
1110 <title>The <filename>do_tst_2+2</> Utility</title>
1111 <para> The shell script <filename>do_tst_2+2</> (in
1112 <filename>$ROOTDIR/tools/ </>)
1113 can be used to check the accuracy of the restart procedure.
1114 </para>
9442059cdd Ed H*1115 </sect3>
1116
1117 </sect2>
fdb5287ce5 Ed H*1118
1119 <sect2 id="packages">
9442059cdd Ed H*1120 <title>Creating MITgcm Packages</title>
fdb5287ce5 Ed H*1121
23e640c5e4 Ed H*1122 <para>Optional parts of code have been separated from the MITgcmUV core
1123 driver code and organised into packages. The packaging structure
1124 provides a mechanism for maintaining suites of code, specific to
1125 particular classes of problems, in a way that is cleanly separated from
1126 the generic fluid dynamical engine.</para>
1127
1128 <para>The MITgcmUV packaging structure is described below using generic
74aa04f48b Jean*1129 package names ${pkg}. A concrete examples of a package is the code for
23e640c5e4 Ed H*1130 implementing GM/Redi mixing. This code uses the package name</para>
fdb5287ce5 Ed H*1131
1132 </sect2>
1133
1134 </sect1>
1135
1136 <sect1>
1137 <title>Chris's Notes...</title>
1138
1139 <programlisting>
1140 MITgcmUV Packages
1141 =================
1142
1143 Optional parts of code are separated from
1144 the MITgcmUV core driver code and organised into
1145 packages. The packaging structure provides a mechanism for
1146 maintaining suites of code, specific to particular
1147 classes of problem, in a way that is cleanly
1148 separated from the generic fluid dynamical engine.
1149
1150 The MITgcmUV packaging structure is describe
1151 below using generic package names ${pkg}.
1152 A concrete examples of a package is the code
1153 for implementing GM/Redi mixing. This code uses
f7a66e0ebc Jean*1154 the package name
fdb5287ce5 Ed H*1155 * ${PKG} = GMREDI
1156 * ${pkg} = gmredi
1157 * ${Pkg} = gmRedi
1158
1159 Package states
1160 ==============
1161
1162 Packages can be any one of four states, included,
1163 excluded, enabled, disabled as follows:
1164
1165 included(excluded) compile time state which
1166 includes(excludes) package
1167 code and routine calls from
1168 compilation/linking etc...
1169
1170 enabled(disabled) run-time state which
1171 enables(disables) package code
1172 execution.
1173
1174 Every call to a ${pkg}_... routine from outside the package
1175 should be placed within both a
1176 #ifdef ALLOW_${PKG} ... block and a
1177 if ( use${Pkg} ) ... then block.
1178 Package states are generally not expected to change during
1179 a model run.
1180
1181 Package structure
1182 =================
1183
1184 o Each package gets its runtime configuration
1185 parameters from a file named "data.${pkg}"
1186 Package runtime config. options are imported
1187 into a common block held in a header file
1188 called "${PKG}.h".
f7a66e0ebc Jean*1189 Note: In some packages, the header file "${PKG}.h" is splitted
1190 into "${PKG}_PARAMS.h" that contains the package parameters and
d866ece84c Jean*1191 ${PKG}_VARS.h" for the field arrays.
fdb5287ce5 Ed H*1192
1193 o The core driver part of the model can check
1194 for runtime enabling or disabling of individual packages
1195 through logical flags use${Pkg}.
1196 The information is loaded from a
1197 global package setup file called "data.pkg".
1198 The use${Pkg} flags are not used within
1199 individual packages.
1200
1201 o Included in "${PKG}.h" is a logical flag
1202 called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
1203 by other packages to check dependencies and requirements
1204 from other packages ( see "Package Boot Sequence" section).
1205 NOTE: This procedure is not presently implemented,
1206 ----- neither for kpp nor for gmRedi.
1207
1208 CPP Flags
1209 =========
1210
1211 1. Within the core driver code flags of the form
1212 ALLOW_${PKG} are used to include or exclude
1213 whole packages. The ALLOW_${PKG} flags are included
f7a66e0ebc Jean*1214 from a PACKAGES_CONFIG.h file that is automatically
d866ece84c Jean*1215 generated by genmake2 (see genmake2 section).
fdb5287ce5 Ed H*1216 held in-line in the CPP_OPTIONS.h header file.
1217 e.g.
1218
1219 Core model code .....
1220
d866ece84c Jean*1221 #include "PACKAGES_CONFIG.h"
fdb5287ce5 Ed H*1222 #include "CPP_OPTIONS.h"
1223 :
1224 :
1225 :
1226
1227 #ifdef ALLOW_${PKG}
1228 if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
1229 #endif
1230
1231 2. Within an individual package a header file,
1232 "${PKG}_OPTIONS.h", is used to set CPP flags
f7a66e0ebc Jean*1233 specific to that package. It also includes
d866ece84c Jean*1234 "PACKAGES_CONFIG.h" and "CPP_OPTIONS.h".
fdb5287ce5 Ed H*1235
1236 Package Boot Sequence
1237 =====================
1238
1239 Calls to package routines within the core code timestepping
1240 loop can vary. However, all packages follow a required
1241 "boot" sequence outlined here:
1242
1243 1. S/R PACKAGES_BOOT()
1244 :
1245 CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
f7a66e0ebc Jean*1246
fdb5287ce5 Ed H*1247 2. S/R PACKAGES_READPARMS()
1248 :
1249 #ifdef ALLOW_${PKG}
1250 if ( use${Pkg} )
1251 & CALL ${PKG}_READPARMS( retCode )
1252 #endif
1253
bbda629aeb Jean*1254 3. S/R PACKAGES_INIT_FIXED()
1255 :
1256 #ifdef ALLOW_${PKG}
1257 if ( use${Pkg} )
1258 & CALL ${PKG}_INIT_FIXED( retCode )
1259 #endif
1260
1261 4. S/R PACKAGES_CHECK()
fdb5287ce5 Ed H*1262 :
1263 #ifdef ALLOW_${PKG}
1264 if ( use${Pkg} )
1265 & CALL ${PKG}_CHECK( retCode )
1266 #else
1267 if ( use${Pkg} )
1268 & CALL PACKAGES_CHECK_ERROR('${PKG}')
1269 #endif
1270
bbda629aeb Jean*1271 5. S/R PACKAGES_INIT_VARIABLES()
fdb5287ce5 Ed H*1272 :
1273 #ifdef ALLOW_${PKG}
1274 if ( use${Pkg} )
bbda629aeb Jean*1275 & CALL ${PKG}_INIT_VARIA( )
fdb5287ce5 Ed H*1276 #endif
1277
bbda629aeb Jean*1278 Package Output
1279 ==============
1280 6. S/R DO_THE_MODEL_IO
1281
1282 #ifdef ALLOW_${PKG}
1283 if ( use${Pkg} )
ad6ab0ddd2 Jean*1284 & CALL ${PKG}_OUTPUT( )
bbda629aeb Jean*1285 #endif
1286
1287 7. S/R PACKAGES_WRITE_PICKUP()
1288
1289 #ifdef ALLOW_${PKG}
1290 if ( use${Pkg} )
1291 & CALL ${PKG}_WRITE_PICKUP( )
1292 #endif
fdb5287ce5 Ed H*1293
1294 Description
1295 ===========
1296
f7a66e0ebc Jean*1297 - ${PKG}_READPARMS()
fdb5287ce5 Ed H*1298 is responsible for reading
1299 in the package parameters file data.${pkg}, and storing
d866ece84c Jean*1300 the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h").
bbda629aeb Jean*1301 -> called from INITIALISE_FIXED in PACKAGES_READPARMS
1302
f7a66e0ebc Jean*1303 - ${PKG}_INIT_FIXED()
1304 is responsible for completing the internal setup of a package.
bbda629aeb Jean*1305 -> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED
1306 note: 1) some pkg use instead:
1307 CALL ${PKG}_INITIALISE ( or the old form CALL ${PKG}_INIT )
1308 2) for simple pkg setup, this part is done inside ${PKG}_READPARMS
fdb5287ce5 Ed H*1309
f7a66e0ebc Jean*1310 - ${PKG}_CHECK()
fdb5287ce5 Ed H*1311 is responsible for validating
1312 basic package setup and inter-package dependencies.
1313 ${PKG}_CHECK can import other package parameters it may
1314 need to check. This is done through header files "${PKG}.h".
1315 It is assumed that parameters owned by other packages
1316 will not be reset during ${PKG}_CHECK().
bbda629aeb Jean*1317 -> called from INITIALISE_FIXED in PACKAGES_CHECK
1318
f7a66e0ebc Jean*1319 - ${PKG}_INIT_VARIA()
bbda629aeb Jean*1320 is responsible for fill-in all package variables with an initial value.
1321 Contains eventually a call to ${PKG}_READ_PICKUP that will read
1322 from a pickup file the package variables required to restart the model.
f7a66e0ebc Jean*1323 This routine is called after the core model state has been completely
bbda629aeb Jean*1324 initialised but before the core model timestepping starts.
1325 -> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES
f7a66e0ebc Jean*1326 note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg
1327 use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old
bbda629aeb Jean*1328 form ${PKG}_INIT
1329
ad6ab0ddd2 Jean*1330 - ${PKG}_OUTPUT( )
d866ece84c Jean*1331 is responsible for writing time-average fields to output files
1332 (but the cumulating step is done within the package main S/R).
f7a66e0ebc Jean*1333 Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR)
1334 and write snap-shot fields that are hold in common blocks. Other
bbda629aeb Jean*1335 temporary fields are directly dump to file where they are available.
f7a66e0ebc Jean*1336 NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages
ad6ab0ddd2 Jean*1337 but is beeing replaced by ${PKG}_OUTPUT
d866ece84c Jean*1338 to avoid confusion with pkg/diagnostics functionality.
1339 2) the output part is not yet in a standard form and might still
1340 evolve a lot.
bbda629aeb Jean*1341 -> called within DO_THE_MODEL_IO
1342
f7a66e0ebc Jean*1343 - ${PKG}_WRITE_PICKUP()
1344 is responsible for writing a package pickup file when necessary for
bbda629aeb Jean*1345 a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT )
1346 -> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP
fdb5287ce5 Ed H*1347
1348 Summary
1349 =======
1350
1351 - CPP options:
1352 -----------------------
1353 * ALLOW_${PKG} include/exclude package for compilation
1354
1355 - FORTRAN logical:
1356 -----------------------
1357 * use${Pkg} enable package for execution at runtime
1358 -> declared in PARAMS.h
1359 * ${Pkg}IsOn for package cross-dependency check
1360 -> declared in ${PKG}.h
1361 N.B.: Not presently used!
1362
1363 - header files
1364 -----------------------
1365 * ${PKG}_OPTIONS.h has further package-specific CPP options
1366 * ${PKG}.h package-specific common block variables, fields
d866ece84c Jean*1367 or ${PKG}_PARAMS.h package-specific common block parameters
1368 and ${PKG}_VARS.h package-specific common block fields
fdb5287ce5 Ed H*1369
1370 - FORTRAN source files
1371 -----------------------
bbda629aeb Jean*1372 * ${pkg}_readparms.F reads parameters from file data.${pkg}
1373 * ${pkg}_init_fixed.F complete the package setup
1374 * ${pkg}_check.F checks package dependencies and consistencies
1375 * ${pkg}_init_varia.F initialises package-related fields
1376 * ${pkg}_... .F package source code
ad6ab0ddd2 Jean*1377 * ${pkg}_output.F write output to file.
bbda629aeb Jean*1378 * ${pkg}_write_pickup.F write a package pickup file to restart the model
fdb5287ce5 Ed H*1379
f7a66e0ebc Jean*1380 New: Subroutine in one package (pkgA) that only contains code which
d866ece84c Jean*1381 is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F)
1382 will be named: pkgA_pkgB_something.F
1383
fdb5287ce5 Ed H*1384 - parameter file
1385 -----------------------
1386 * data.${pkg} parameter file
1387 </programlisting>
1388
1389 </sect1>
8eab949e1c Ed H*1390
74aa04f48b Jean*1391 <sect1 id="documentation">
1392 <title>Editing the Documentation</title>
1393
1394 <sect2 id="documentation_getting">
1395 <title>Getting the Docs and Code</title>
1396
1397 <para>The first step towards editing the documentation is to checkout a
1398 copy of code, docs, and build scripts from the CVS server using:</para>
1399
1400 <screen>
1401 $ export CVS_RSH=ssh
1402 $ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack'
1403 $ mkdir scratch
1404 $ cvs co -P MITgcm manual mitgcm.org
1405 </screen>
1406
1407 <para>These commands extract the necessary information from the CVS server
1408 and create a temporary (called <filename>scratch</filename>) directory for
1409 the storage of the HTML and other files that will be created. Please note
1410 that you must either create <filename>scratch</filename> as shown or edit
1411 the various <filename>Makefile</filename>s and scripts used to create the
1412 documentation.</para>
1413 </sect2>
1414
1415 <sect2>
1416 <title>Editing the Documentation</title>
1417
1418 <para>The documentation is contained in the <filename>manual</filename>
1419 directory in a raw LaTeX format. The main document is
1420 <filename>manual.tex</filename> and it uses <command>\input{}</command>s
1421 to include the chapters and subsections.</para>
1422
1423 <para>Since the same LaTeX source is used to produce PostScript, PDF, and
1424 HTML output, care should be taken to follow certain conventions. Two of
1425 the most important are the usage of the <command>\filelink{}{}</command>
1426 and <command>\varlink{}{}</command> commands. Both of these commands have
1427 been defined to simplify the connection between the automatically
1428 generated ("code browser") HTML and the HTML version of the manual
1429 produced by LaTeX2HTML. They each take two arguments (corresponding to
1430 the contents of the two sets of curly braces) which are the text that the
1431 author wishes to be "wrapped" within the link, and a specially formatted
1432 link thats relative to the <filename>MITgcm</filename> directory within
1433 the CVS tree.</para>
1434
1435 <para>The result is a command that resembles either</para>
f7a66e0ebc Jean*1436
74aa04f48b Jean*1437 <orderedlist>
1438 <listitem>
1439 <para>a reference to a variable or subroutine name such as
1440 <command>\varlink{tRef}{tRef}</command>, or </para>
1441 </listitem>
1442
1443 <listitem>
1444 <para>a reference to a file such as
1445 <command>\varlink{tRef}{path-to-the-file_name.F}</command>
1446 where the absolute path to the file is of the form
1447 <filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>
1448 <para>(please note how the leading "/foo/MITgcm"
1449 component of the path is dropped leaving the path
1450 <emphasis>relative</emphasis> to the head of the code
1451 directory and each directory separator "/" is turned
1452 into a "-")</para>
1453 </listitem>
1454 </orderedlist>
f7a66e0ebc Jean*1455
74aa04f48b Jean*1456
1457 </sect2>
1458
1459 <sect2>
1460 <title>Building the Documentation</title>
f7a66e0ebc Jean*1461
74aa04f48b Jean*1462 <para>Given the directory structure of <xref
1463 linkend="documentation_getting">, the entire documentation for the web
1464 site can be built using:</para>
1465
1466 <screen>
1467 $ cd mitgcm.org/devel/buildweb
1468 $ make All
1469 </screen>
1470
1471 <para>Which builds the PDF from the LaTeX source, creates the HTML output
1472 from the LaTeX source, parses the FORTRAN code base to produce a
1473 hyperlinked HTML version of the source, and then determines the
1474 cross-linking between the various HTML components.</para>
1475
1476 <para>If there are no errors, the result of the build process (which can
1477 take 30+ minutes on a P4/2.5Ghz) will be contained within a single
1478 directory called <filename>scratch/dev_docs</filename>. This is a freshly
1479 built version of the entire on-line users manual. If you have the correct
1480 permissions, it can be directly copied to the web server area:</para>
1481
1482 <screen>
1483 $ mv scratch/dev_docs /u/u0/httpd/html
1484 </screen>
1485
1486 <para>and the update is complete.</para>
1487
1488 </sect2>
1489
1490 </sect1>
1491
8eab949e1c Ed H*1492 </article>
1493
