MITgcm |
|
|
|
|||
File indexing completed on 2022-04-19 05:08:38 UTC
view on githubraw file Latest commit a85293d0 on 2022-04-18 21:18:22 UTC4c563c2ee9 Chri*0001 CBOP 8df2d31323 Jean*0002 C !ROUTINE: EESUPPORT.h 4c563c2ee9 Chri*0003 C !INTERFACE: 0004 C include "EESUPPORT.h" 924557e60a Chri*0005 C 4c563c2ee9 Chri*0006 C !DESCRIPTION: 0007 C *==========================================================* 924557e60a Chri*0008 C | EESUPPORT.h | 4c563c2ee9 Chri*0009 C *==========================================================* a85293d087 Mart*0010 C | Support data structures for the MITgcm UV | 0011 C | "execution environment" code. This data should be | 0012 C | private to the execution environment routines. Data | 0013 C | which needs to be accessed directly by a numerical model | 0014 C | goes in EEPARAMS.h. | 4c563c2ee9 Chri*0015 C *==========================================================* 0016 CEOP 924557e60a Chri*0017 0018 C ERROR_HEADER - String which prefixes error messages 0019 CHARACTER*(*) ERROR_HEADER 0020 PARAMETER ( ERROR_HEADER = ' *** ERROR ***' ) 0021 C PROCESS_HEADER - String which prefixes processor number 0022 CHARACTER*(*) PROCESS_HEADER 0023 PARAMETER ( PROCESS_HEADER = 'PID.TID' ) 0024 a85d6ab24e Chri*0025 C MAX_NUM_COMM_MODES - Maximum number of communication modes 924557e60a Chri*0026 C COMM_NONE - No edge communication a85d6ab24e Chri*0027 C COMM_MSG - Use messages to communicate edges 0028 C COMM_PUT - Use put to communicate edges 0029 C COMM_GET - Use get to communicate edges 0030 C Note - commName holds an identifying name for each communication 0031 C mode. The COMM_ parameters are used to index commName 0032 C so the COMM_ parameters need to be in the range 0033 C 1 : MAX_NUM_COMM_MODES. 0034 INTEGER MAX_NUM_COMM_MODES 0035 PARAMETER ( MAX_NUM_COMM_MODES = 4 ) 924557e60a Chri*0036 INTEGER COMM_NONE a85d6ab24e Chri*0037 PARAMETER ( COMM_NONE = 1 ) 0038 INTEGER COMM_MSG 0039 PARAMETER ( COMM_MSG = 2 ) 0040 INTEGER COMM_PUT 0041 PARAMETER ( COMM_PUT = 3 ) 0042 INTEGER COMM_GET 0043 PARAMETER ( COMM_GET = 4 ) 0044 COMMON /EESUPP_COMMNAME/ commName 0045 CHARACTER*10 commName(MAX_NUM_COMM_MODES) 0046 0047 C Tile identifiers 0048 C Tiles have a number that is unique over the global domain. 0049 C A tile that is not there has its number set to NULL_TILE 0050 INTEGER NULL_TILE 0051 PARAMETER ( NULL_TILE = -1 ) 0052 924557e60a Chri*0053 0054 C-- COMMON /EESUPP_C/ Execution environment support character variables 0055 C myProcessStr - String identifying my process number 0056 COMMON /EESUPP_C/ myProcessStr 0057 CHARACTER*128 myProcessStr 0058 0059 C-- COMMON /EESUPP_L/ Execution environment support logical variables 0060 C initMPError - Flag indicating error during multi-processing 0061 C initialisation. 0062 C finMPError - Flag indicating error during multi-processing 0063 C termination. 0064 C ThError - Thread detected an error. 0065 C usingMPI - Flag controlling use of MPI routines. This flag 0066 C allows either MPI or threads to be used in a 0067 C shared memory environment which can be a useful 0068 C debugging/performance analysis tool. 0069 C usingSyncMessages - Flag that causes blocking communication to be used 0070 C if possible. When false non-blocking EXCH routines 0071 C will be used if possible. 0072 C notUsingXPeriodicity - Flag indicating no X/Y boundary wrap around 0073 C notUsingYPeriodicity This affects the communication routines but 0074 C is generally ignored in the numerical model 0075 C code. 0076 C threadIsRunning, threadIsComplete - Flags used to check for correct behaviour 0077 C of multi-threaded code. 0078 C threadIsRunning is used to check that the 0079 C threads we need are running. This catches the 0080 C situation where a program eedata file has nTthreads 0081 C greater than the setenv PARALLEL or NCPUS variable. 0082 C threadIsComplete is used to flag that a thread has 0083 C reached the end of the model. This is used as a check to 0084 C trap problems that might occur if one thread "escapes" 0085 C the main.F master loop. This should not happen 0086 C if the multi-threading compilation tools works right. 0087 C But (see for example KAP) this is not always the case! 0088 COMMON /EESUPP_L/ thError, threadIsRunning, threadIsComplete, 0089 & allMyEdgesAreSharedMemory, usingMPI, usingSyncMessages, c36c6d9b4f Jean*0090 & notUsingXPeriodicity, notUsingYPeriodicity 924557e60a Chri*0091 LOGICAL thError(MAX_NO_THREADS) 0092 LOGICAL threadIsRunning(MAX_NO_THREADS) 0093 LOGICAL threadIsComplete(MAX_NO_THREADS) 0094 LOGICAL allMyEdgesAreSharedMemory(MAX_NO_THREADS) 0095 LOGICAL usingMPI 0096 LOGICAL usingSyncMessages 0097 LOGICAL notUsingXPeriodicity 0098 LOGICAL notUsingYPeriodicity c36c6d9b4f Jean*0099 924557e60a Chri*0100 C-- COMMON /EESUPP_I/ Parallel support integer globals a85d6ab24e Chri*0101 C pidW - Process ID of neighbor to West 0102 C pidE - ditto East 0103 C pidN - ditto North 0104 C pidS - ditto South 924557e60a Chri*0105 C Note: pid[XY] is not necessairily the UNIX 0106 C process id - it is just an identifying 0107 C number. a85d6ab24e Chri*0108 C myPid - My own process id 0109 C nProcs - Number of processes 0110 C westCommunicationMode - Mode of communication for each tile face 0111 C eastCommunicationMode 0112 C northCommunicationMode 0113 C southCommunicationMode 0114 C bi0 - Low cartesian tile index for this process 0115 C bj0 Note - In a tile distribution with holes bi0 and bj0 0116 C are not useful. Neighboring tile indices must 0117 C be derived some other way. 0118 C tileNo - Tile identification number for my tile and 0119 C tileNo[WENS] my N,S,E,W neighbor tiles. 0120 C tilePid[WENS] - Process identification number for 0121 C my N,S,E,W neighbor tiles. 0122 C nTx, nTy - No. threads in X and Y. This assumes a simple 0123 C cartesian gridding of the threads which is not 0124 C required elsewhere but that makes it easier. 924557e60a Chri*0125 COMMON /EESUPP_I/ a85d6ab24e Chri*0126 & myPid, nProcs, pidW, pidE, pidN, pidS, 0127 & tileCommModeW, tileCommModeE, 0128 & tileCommModeN, tileCommModeS, 0129 & tileNo, tileNoW, tileNoE, tileNoS, tileNoN, 0130 & tilePidW, tilePidE, tilePidS, tilePidN, 0131 & tileBiW, tileBiE, tileBiS, tileBiN, 0132 & tileBjW, tileBjE, tileBjS, tileBjN, 0133 & tileTagSendW, tileTagSendE, tileTagSendS, tileTagSendN, 0134 & tileTagRecvW, tileTagRecvE, tileTagRecvS, tileTagRecvN 0135 INTEGER myPid 0136 INTEGER nProcs 924557e60a Chri*0137 INTEGER pidW 0138 INTEGER pidE 0139 INTEGER pidN a85d6ab24e Chri*0140 INTEGER pidS 0141 INTEGER tileCommModeW ( nSx, nSy ) 0142 INTEGER tileCommModeE ( nSx, nSy ) 0143 INTEGER tileCommModeN ( nSx, nSy ) 0144 INTEGER tileCommModeS ( nSx, nSy ) 0145 INTEGER tileNo( nSx, nSy ) 0146 INTEGER tileNoW( nSx, nSy ) 0147 INTEGER tileNoE( nSx, nSy ) 0148 INTEGER tileNoN( nSx, nSy ) 0149 INTEGER tileNoS( nSx, nSy ) 0150 INTEGER tilePidW( nSx, nSy ) 0151 INTEGER tilePidE( nSx, nSy ) 0152 INTEGER tilePidN( nSx, nSy ) 0153 INTEGER tilePidS( nSx, nSy ) 0154 INTEGER tileBiW( nSx, nSy ) 0155 INTEGER tileBiE( nSx, nSy ) 0156 INTEGER tileBiN( nSx, nSy ) 0157 INTEGER tileBiS( nSx, nSy ) 0158 INTEGER tileBjW( nSx, nSy ) 0159 INTEGER tileBjE( nSx, nSy ) 0160 INTEGER tileBjN( nSx, nSy ) 0161 INTEGER tileBjS( nSx, nSy ) 0162 INTEGER tileTagSendW( nSx, nSy ) 0163 INTEGER tileTagSendE( nSx, nSy ) 0164 INTEGER tileTagSendN( nSx, nSy ) 0165 INTEGER tileTagSendS( nSx, nSy ) 0166 INTEGER tileTagRecvW( nSx, nSy ) 0167 INTEGER tileTagRecvE( nSx, nSy ) 0168 INTEGER tileTagRecvN( nSx, nSy ) 0169 INTEGER tileTagRecvS( nSx, nSy ) 924557e60a Chri*0170 0171 #ifdef ALLOW_USE_MPI 0172 C-- Include MPI standard Fortran header file 0173 #include "mpif.h" 0174 #define _mpiTRUE_ 1 0175 #define _mpiFALSE_ 0 0176 0177 C-- COMMON /EESUPP_MPI_I/ MPI parallel support integer globals 0178 C mpiPidW - MPI process id for west neighbor. 0179 C mpiPidE - MPI process id for east neighbor. 0180 C mpiPidN - MPI process id for north neighbor. 0181 C mpiPidS - MPI process id for south neighbor. 0182 C mpiPidNW - MPI process id for northwest neighbor. 0183 C mpiPidNE - MPI process id for northeast neighbor. 0184 C mpiPidSW - MPI process id for southwest neighbor. 0185 C mpiPidSE - MPI process id for southeast neighbor. 0186 C mpiPidIO - MPI process to use for IO. 0187 C mpiNprocs - No. of MPI processes. 0188 C mpiMyId - MPI process id of me. 0189 C mpiComm - MPI communicator to use. 0190 C mpiPx - My MPI proc. grid X coord 0191 C mpiPy - My MPI proc. grid Y coord 0192 C mpiXGlobalLo - My bottom-left (south-west) x-coordinate in 0193 C global domain. 0194 C mpiYGlobalLo - My bottom-left (south-west) y-coordinate in 0195 C global domain. 0196 C mpiTypeXFaceBlock_xy_r4 - Primitives for communicating edge 0197 C mpiTypeXFaceBlock_xy_r8 of a block. 0198 C mpiTypeYFaceBlock_xy_r4 XFace is used in east-west transfer 0199 C mpiTypeYFaceBlock_xy_r8 YFace is used in nrth-south transfer 0200 C mpiTypeXFaceBlock_xyz_r4 xy is used in two-dimensional arrays 0201 C mpiTypeXFaceBlock_xyz_r8 xyz is used with three-dimensional arrays 0202 C mpiTypeYFaceBlock_xyz_r4 r4 is used for real*4 data 0203 C mpiTypeYFaceBlock_xyz_r8 r8 is used for real*8 data 0204 C mpiTypeXFaceThread_xy_r4 - Composites of the above primitives 0205 C mpiTypeXFaceThread_xy_r8 for communicating edges of all blocks 0206 C mpiTypeYFaceThread_xy_r4 owned by a thread. 0207 C mpiTypeYFaceThread_xy_r8 0208 C mpiTypeXFaceThread_xyz_r4 0209 C mpiTypeXFaceThread_xyz_r8 0210 C mpiTypeYFaceThread_xyz_r4 0211 C mpiTypeYFaceBlock_xyz_r8 0212 C mpiTagE - Tags are needed to mark requests when MPI is running 0213 C mpiTagW between multithreaded processes or when the same process. 0214 C mpiTagS is a neighbor in more than one direction. The tags ensure that 0215 C mpiTagN a thread will get the message it is looking for. 0216 C mpiTagSW The scheme adopted is to tag messages according to 0217 C mpiTagSE the direction they are travelling. Thus a message 0218 C mpiTagNW travelling east is tagged mpiTagE. However, in a 0219 C mpiTagNE multi-threaded environemnt several messages could 0220 C be travelling east from the same process at the 0221 C same time. The tag is therefore modified to 0222 C be mpiTag[EWS...]*nThreads+myThid. This requires that 0223 C each thread also know the thread ids of its "neighbor" 0224 C threads. 0225 COMMON /EESUPP_MPI_I/ 0226 & mpiPidW, mpiPidE, mpiPidS, mpiPidN, 0227 & mpiPidSE, mpiPidSW, mpiPidNE, mpiPidNW, 0228 & mpiPidIo, mpiMyId, mpiNProcs, mpiComm, 0229 & mpiPx, mpiPy, mpiXGlobalLo, mpiYGlobalLo, 0230 & mpiTypeXFaceBlock_xy_r4, mpiTypeXFaceBlock_xy_r8, 0231 & mpiTypeYFaceBlock_xy_r4, mpiTypeYFaceBlock_xy_r8, 0232 & mpiTypeXFaceBlock_xyz_r4, mpiTypeXFaceBlock_xyz_r8, 0233 & mpiTypeYFaceBlock_xyz_r4, mpiTypeYFaceBlock_xyz_r8, 0234 & mpiTypeXFaceThread_xy_r4, mpiTypeXFaceThread_xy_r8, 0235 & mpiTypeYFaceThread_xy_r4, mpiTypeYFaceThread_xy_r8, 0236 & mpiTypeXFaceThread_xyz_r4, mpiTypeXFaceThread_xyz_r8, 0237 & mpiTypeYFaceThread_xyz_r4, mpiTypeYFaceThread_xyz_r8, 0238 & mpiTagE, mpiTagW, mpiTagN, mpiTagS, 0239 & mpiTagSE, mpiTagSW, mpiTagNW, mpiTagNE 0240 0241 INTEGER mpiPidW 0242 INTEGER mpiPidE 0243 INTEGER mpiPidS 0244 INTEGER mpiPidN 0245 INTEGER mpiPidSW 0246 INTEGER mpiPidSE 0247 INTEGER mpiPidNW 0248 INTEGER mpiPidNE 0249 INTEGER mpiPidIO 0250 INTEGER mpiMyId 0251 INTEGER mpiNProcs 0252 INTEGER mpiComm 0253 INTEGER mpiPx 0254 INTEGER mpiPy 0255 INTEGER mpiXGlobalLo 0256 INTEGER mpiYGlobalLo 0257 INTEGER mpiTypeXFaceBlock_xy_r4 0258 INTEGER mpiTypeXFaceBlock_xy_r8 0259 INTEGER mpiTypeYFaceBlock_xy_r4 0260 INTEGER mpiTypeYFaceBlock_xy_r8 0261 INTEGER mpiTypeXFaceBlock_xyz_r4 0262 INTEGER mpiTypeXFaceBlock_xyz_r8 0263 INTEGER mpiTypeYFaceBlock_xyz_r4 0264 INTEGER mpiTypeYFaceBlock_xyz_r8 0265 INTEGER mpiTypeXFaceThread_xy_r4(MAX_NO_THREADS) 0266 INTEGER mpiTypeXFaceThread_xy_r8(MAX_NO_THREADS) 0267 INTEGER mpiTypeYFaceThread_xy_r4(MAX_NO_THREADS) 0268 INTEGER mpiTypeYFaceThread_xy_r8(MAX_NO_THREADS) 0269 INTEGER mpiTypeXFaceThread_xyz_r4(MAX_NO_THREADS) 0270 INTEGER mpiTypeXFaceThread_xyz_r8(MAX_NO_THREADS) 0271 INTEGER mpiTypeYFaceThread_xyz_r4(MAX_NO_THREADS) 0272 INTEGER mpiTypeYFaceThread_xyz_r8(MAX_NO_THREADS) 0273 INTEGER mpiTagNW 0274 INTEGER mpiTagNE 0275 INTEGER mpiTagSW 0276 INTEGER mpiTagSE 0277 INTEGER mpiTagW 0278 INTEGER mpiTagE 0279 INTEGER mpiTagN 0280 INTEGER mpiTagS 0281 8df2d31323 Jean*0282 C-- COMMON /MPI_FULLMAP_I/ holds integer arrays of the full list of MPI process 0283 C mpi_myXGlobalLo :: List of all processors bottom-left X-index in global domain 0284 C mpi_myYGlobalLo :: List of all processors bottom-left Y-index in global domain 0285 C Note: needed for mpi gather/scatter routines & singleCpuIO. 0286 COMMON /MPI_FULLMAP_I/ 0287 & mpi_myXGlobalLo, mpi_myYGlobalLo 0288 INTEGER mpi_myXGlobalLo(nPx*nPy) 0289 INTEGER mpi_myYGlobalLo(nPx*nPy) 0290 9d9b5e8eba Alis*0291 C MPI communicator describing this model realization 0292 COMMON /MPI_COMMS/ 0293 & MPI_COMM_MODEL 0294 INTEGER MPI_COMM_MODEL 8df2d31323 Jean*0295 9d9b5e8eba Alis*0296 #endif /* ALLOW_USE_MPI */
| [ Source navigation ] | [ Diff markup ] | [ Identifier search ] | [ general search ] |
|
This page was automatically generated from https://github.com/MITgcm/MITgcm by the 2.2.1-MITgcm-0.1 LXR engine. The LXR team |
|