File boutmesh.hxx

class BoutMesh : public Mesh

#include <boutmesh.hxx>

Implementation of Mesh (mostly) compatible with BOUT

Topology and communications compatible with BOUT conventions.

Public Functions

BoutMesh(GridDataSource *s, Options *options = nullptr)

~BoutMesh() override

virtual int load() override

Read in the mesh from data sources.

virtual comm_handle send(FieldGroup &g) override

Send data between processors Does not wait for communications to complete, so wait() must be called before guard cell values are used

Example

comm_handle handle = mesh->send(group); … mesh->wait(handle);

Parameters:

g – [in] A group of fields to communicate

virtual comm_handle sendX(FieldGroup &g, comm_handle handle = nullptr, bool disable_corners = false) override

Send only in the x-direction.

virtual comm_handle sendY(FieldGroup &g, comm_handle handle = nullptr) override

Send only in the y-direction.

virtual int wait(comm_handle handle) override

Wait for a send operation to complete

Parameters:

handle – [in] The handle returned by send()

virtual int getNXPE() override

The number of processors in the X direction.

virtual int getNYPE() override

The number of processors in the Y direction.

virtual int getXProcIndex() override

This processor’s index in X direction.

virtual int getYProcIndex() override

This processor’s index in Y direction.

virtual bool firstX() const override

Is this processor the first in X? i.e. is there a boundary to the left in X?

virtual bool lastX() const override

Is this processor last in X? i.e. is there a boundary to the right in X?

virtual int sendXOut(BoutReal *buffer, int size, int tag) override

Send a buffer of data to processor at X index +1

Parameters:

• buffer – [in] The data to send. Must be at least length size

• size – [in] The number of BoutReals to send

• tag – [in] A label for the communication. Must be the same at receive

virtual int sendXIn(BoutReal *buffer, int size, int tag) override

Send a buffer of data to processor at X index -1

Parameters:

• buffer – [in] The data to send. Must be at least length size

• size – [in] The number of BoutReals to send

• tag – [in] A label for the communication. Must be the same at receive

virtual comm_handle irecvXOut(BoutReal *buffer, int size, int tag) override

Receive a buffer of data from X index +1

Parameters:

• buffer – [in] A buffer to put the data in. Must already be allocated of length size

• size – [in] The number of BoutReals to receive and put in buffer

• tag – [in] A label for the communication. Must be the same as sent

virtual comm_handle irecvXIn(BoutReal *buffer, int size, int tag) override

Receive a buffer of data from X index -1

Parameters:

• buffer – [in] A buffer to put the data in. Must already be allocated of length size

• size – [in] The number of BoutReals to receive and put in buffer

• tag – [in] A label for the communication. Must be the same as sent

inline virtual MPI_Comm getXcomm(int jy) const override

Return communicator containing all processors in X.

virtual MPI_Comm getYcomm(int xpos) const override

Return communicator containing all processors in Y.

virtual bool periodicY(int jx, BoutReal &ts) const override

Is local X index jx periodic in Y?

Parameters:

• jx – [in] The local (on this processor) index in X

• ts – [out] The Twist-Shift angle if periodic

virtual bool periodicY(int jx) const override

Is local X index jx periodic in Y?

Parameters:

jx – [in] The local (on this processor) index in X

virtual int numberOfYBoundaries() const override

Get number of boundaries in the y-direction, i.e. locations where there are boundary cells in the global grid

virtual std::pair<bool, BoutReal> hasBranchCutLower(int jx) const override

Is there a branch cut at this processor’s lower boundary?

Parameters:

jx – [in] The local (on this processor) index in X

Returns:

pair<bool, BoutReal> - bool is true if there is a branch cut, BoutReal gives the total zShift for a 2pi poloidal circuit if there is a branch cut

virtual std::pair<bool, BoutReal> hasBranchCutUpper(int jx) const override

Is there a branch cut at this processor’s upper boundary?

Parameters:

jx – [in] The local (on this processor) index in X

Returns:

pair<bool, BoutReal> - bool is true if there is a branch cut, BoutReal gives the total zShift for a 2pi poloidal circuit if there is a branch cut

virtual int ySize(int xpos) const override

The number of points in Y at fixed X index jx.

virtual bool firstY() const override

Is this processor first in Y? Note: First on the global grid, not necessarily at a boundary

virtual bool lastY() const override

Is this processor last in Y? Note: Last on the global grid, not necessarily at a boundary

virtual bool firstY(int xpos) const override

Is this processor first in Y? Note: Not necessarily at a boundary, but first in the Y communicator for the flux surface through local X index xpos

virtual bool lastY(int xpos) const override

Is this processor last in Y? Note: Not necessarily at a boundary, but last in the Y communicator for the flux surface through local X index xpos

virtual RangeIterator iterateBndryLowerY() const override

Iterate over the lower Y boundary.

virtual RangeIterator iterateBndryUpperY() const override

Iterate over the upper Y boundary.

virtual RangeIterator iterateBndryLowerInnerY() const override

virtual RangeIterator iterateBndryLowerOuterY() const override

virtual RangeIterator iterateBndryUpperInnerY() const override

virtual RangeIterator iterateBndryUpperOuterY() const override

virtual std::vector<BoundaryRegion*> getBoundaries() override

Return a vector containing all the boundary regions on this processor.

virtual std::vector<std::shared_ptr<BoundaryRegionPar>> getBoundariesPar(BoundaryParType type) override

Get the list of parallel boundary regions. The option specifies with region to get. Default is to get all regions. All possible options are listed at the top of this file, see BoundaryParType. For example: get all regions: mesh->getBoundariesPar(Mesh::BoundaryParType::all) get only xout: mesh->getBoundariesPar(Mesh::BoundaryParType::xout)

virtual void addBoundaryPar(std::shared_ptr<BoundaryRegionPar> bndry, BoundaryParType type) override

Add a parallel(Y) boundary to this processor.

virtual std::set<std::string> getPossibleBoundaries() const override

Get the set of all possible boundaries in this configuration.

virtual Field3D smoothSeparatrix(const Field3D &f) override

Branch-cut special handling (experimental)

inline int getNx() const

inline int getNy() const

virtual BoutReal GlobalX(int jx) const override

Continuous X index between 0 and 1.

virtual BoutReal GlobalY(int jy) const override

Continuous Y index (0 -> 1)

virtual BoutReal GlobalX(BoutReal jx) const override

Continuous X index between 0 and 1.

virtual BoutReal GlobalY(BoutReal jy) const override

Continuous Y index (0 -> 1)

inline BoutReal getIxseps1() const

inline BoutReal getIxseps2() const

void outputVars(Options &output_options) override

virtual int getGlobalXIndex(int xlocal) const override

Returns a global X index given a local index. Global index includes boundary cells, local index includes boundary or guard cells.

virtual int getGlobalXIndexNoBoundaries(int xlocal) const override

Returns a global X index given a local index. Global index excludes boundary cells, local index includes boundary or guard cells.

virtual int getLocalXIndex(int xglobal) const override

Returns a local X index given a global index. Global index includes boundary cells, local index includes boundary or guard cells.

virtual int getLocalXIndexNoBoundaries(int xglobal) const override

Returns a local X index given a global index. Global index excludes boundary cells, local index includes boundary or guard cells.

virtual int getGlobalYIndex(int ylocal) const override

Returns a global Y index given a local index. Global index includes boundary cells, local index includes boundary or guard cells.

virtual int getGlobalYIndexNoBoundaries(int ylocal) const override

Returns a global Y index given a local index. Global index excludes boundary cells, local index includes boundary or guard cells.

virtual int getLocalYIndex(int yglobal) const override

Returns a local Y index given a global index. Global index includes boundary cells, local index includes boundary or guard cells.

virtual int getLocalYIndexNoBoundaries(int yglobal) const override

Returns a local Y index given a global index. Global index excludes boundary cells, local index includes boundary or guard cells.

virtual int getGlobalZIndex(int zlocal) const override

Returns a global Z index given a local index. Global index includes boundary cells, local index includes boundary or guard cells.

virtual int getGlobalZIndexNoBoundaries(int zlocal) const override

Returns a global Z index given a local index. Global index excludes boundary cells, local index includes boundary or guard cells.

virtual int getLocalZIndex(int zglobal) const override

Returns a local Z index given a global index. Global index includes boundary cells, local index includes boundary or guard cells.

virtual int getLocalZIndexNoBoundaries(int zglobal) const override

Returns a local Z index given a global index. Global index excludes boundary cells, local index includes boundary or guard cells.

Protected Functions

BoutMesh(int input_nx, int input_ny, int input_nz, int mxg, int myg, int nxpe, int nype, int pe_xind, int pe_yind, bool create_topology = true, bool symmetric_X = true, bool symmetric_Y = true)

A constructor used when making fake meshes for testing. This will make a mesh which thinks it corresponds to the subdomain on one processor, even though it’s actually being run in serial.

Pass create_topology = false to not set up topology, regions etc.

BoutMesh(int input_nx, int input_ny, int input_nz, int mxg, int myg, int nxpe, int nype, int pe_xind, int pe_yind, bool symmetric_X, bool symmetric_Y, bool periodic_X, int ixseps1_, int ixseps2_, int jyseps1_1_, int jyseps2_1_, int jyseps1_2_, int jyseps2_2_, int ny_inner_, bool create_regions = true)

Another constructor useful for testing, and used in getPossibleBoundaries. create_regions controls whether or not the various Regions are created on the new mesh

inline BoutMesh(int input_nx, int input_ny, int input_nz, int mxg, int myg, int input_npes)

Very basic initialisation, only suitable for testing.

void overlapHandleMemory(BoutMesh *yup, BoutMesh *ydown, BoutMesh *xin, BoutMesh *xout)

For debugging purposes (when creating fake parallel meshes), make the send and receive buffers share memory. This allows for communications to be faked between meshes as though they were on different processors.

void setYDecompositionIndices(int jyseps1_1_, int jyseps2_1_, int jyseps1_2_, int jyseps2_2_, int ny_inner_)

Set the various y-decomposition indices, enforcing the following invariants:

• jyseps1_1 >= -1

• jyseps2_1 >= jyseps1_1

• jyseps1_2 >= jyseps2_1

• jyseps2_2 >= jyseps1_2

• jyseps2_2 < ny

Inputs inconsistent with these invariants will be set to the minimum acceptable value.

Also sets numberOfXPoints consistently (0, 1, or 2).

YDecompositionIndices setYDecompositionIndices(const YDecompositionIndices &indices)

Version of setYDecompositionindices that returns the values used, useful for testing

void chooseProcessorSplit(Options &options)

Choose NXPE (or NYPE) based on user input.

void findProcessorSplit()

Find a value for NXPE.

void setXDecompositionIndices(const XDecompositionIndices &indices)

Set the two x-decomposition indices. No invariants are enforced.

void setDerivedGridSizes()

Set the derived grid sizes like LocalN*, GlobalN*, Offset*, etc

Requires the following to be set first:

• nx, ny, nz

• MXG, MYG, MZG

• NXPE, NYPE, NZPE

• PE_XIND, PE_YIND

• jyseps1_2, jyseps2_1

void createCommunicators()

Create the various sub-communicators.

void createXBoundaries()

Create the boundary regions in X.

void createYBoundaries()

Create the boundary regions in Y.

void setShiftAngle(const std::vector<BoutReal> &shift_angle)

Set the shift angle and enable twist shift. Should only be used for testing!

int PROC_NUM(int xind, int yind) const

Returns the processor number, given X (xind) and Y (yind) processor indices. Returns -1 if out of range (no processor)

int YGLOBAL(int yloc, int yproc) const

int YLOCAL(int yglo, int yproc) const

int YPROC(int yind) const

Return the Y processor number given a global Y index.

int XPROC(int xind) const

Return the X processor number given a global X index.

inline ConnectionInfo getConnectionInfo() const

Return the communication parameters as calculated by topology

void default_connections()

Connection initialisation: Set processors in a simple 2D grid.

void set_connection(int ypos1, int ypos2, int xge, int xlt, bool ts = false)

Add a topology connection

Set ypos1 and ypos2 to be neighbours in the range xge <= x < xlt. Optional argument ts sets whether to use twist-shift condition

void add_target(int ypos, int xge, int xlt)

Add a divertor target or limiter

ypos is the y index which will become an upper target. ypos+1 will become a lower target. Target created in the range xge <= x < xlt.

void topology()

Create the communication connections between processors to set the layout of the global grid. See the manual section on “BOUT++

Topology” for more information

void addBoundaryRegions()

Adds 2D and 3D regions for boundaries.

Private Functions

int XGLOBAL(BoutReal xloc, BoutReal &xglo) const

Returns the global X index given a local index.

int YGLOBAL(BoutReal yloc, BoutReal &yglo) const

void free_handle(CommHandle *h)

CommHandle *get_handle(int xlen, int ylen)

void clear_handles()

void post_receiveX(CommHandle &ch)

Create the MPI requests to receive data in the x-direction. Non-blocking call.

void post_receiveY(CommHandle &ch)

Create the MPI requests to receive data in the y-direction. Non-blocking call.

int pack_data(const std::vector<FieldData*> &var_list, int xge, int xlt, int yge, int ylt, BoutReal *buffer)

Take data from objects and put into a buffer.

int unpack_data(const std::vector<FieldData*> &var_list, int xge, int xlt, int yge, int ylt, BoutReal *buffer)

Copy data from a buffer back into the fields.

Private Members

std::string gridname

int nx

int ny

int nz

Size of the grid in the input file.

int MX

int MY

int MZ

size of the grid excluding boundary regions

int MYSUB

int MXSUB

int MZSUB

Size of the grid on this processor.

int NPES

Number of processors.

int MYPE

Rank of this processor.

int PE_YIND

Y index of this processor.

int NYPE

int NZPE

bool MYPE_IN_CORE = {false}

Is this processor in the core region?

int ixseps1

int ixseps2

int jyseps1_1

int jyseps2_1

int jyseps1_2

int jyseps2_2

int ixseps_inner

int ixseps_outer

int ixseps_upper

int ixseps_lower

int ny_inner

std::vector<BoutReal> ShiftAngle

Angle for twist-shift location.

bool TS_up_in

bool TS_up_out

bool TS_down_in

bool TS_down_out

int UDATA_INDEST

int UDATA_OUTDEST

int UDATA_XSPLIT

int DDATA_INDEST

int DDATA_OUTDEST

int DDATA_XSPLIT

int IDATA_DEST

int ODATA_DEST

bool TwistShift

bool symmetricGlobalX

Use a symmetric definition in GlobalX() function.

bool symmetricGlobalY

int zperiod

BoutReal ZMIN

BoutReal ZMAX

int MXG

int MYG

int MZG

std::string grid_id = ""

std::string hypnotoad_version = ""

std::string hypnotoad_git_hash = ""

std::string hypnotoad_git_diff = ""

std::string hypnotoad_geqdsk_filename = ""

std::vector<BoundaryRegion*> boundary

std::array< std::vector< std::shared_ptr< BoundaryRegionPar > >, static_cast< int >BoundaryParType::SIZE)> par_boundary

bool async_send = {false}

Switch to asyncronous sends (ISend, not Send)

std::list<CommHandle*> comm_list

MPI_Comm comm_x = {MPI_COMM_NULL}

Communicator containing all processors in X.

MPI_Comm comm_inner = {MPI_COMM_NULL}

Communicator in Y inside both separatrices.

MPI_Comm comm_middle = {MPI_COMM_NULL}

Communicator in Y between separatrices.

MPI_Comm comm_outer = {MPI_COMM_NULL}

Communicator in Y outside both separatrices.

struct CommHandle

Communication handle Used to keep track of communications between send and receive

Public Members

MPI_Request request[6]

Array of receive requests. One for each possible neighbour; one each way in X, two each way in Y

MPI_Request sendreq[6]

Array of send requests (for non-blocking send). One for each possible neighbour; one each way in X, two each way in Y

int xbufflen

Length of the buffers used to send/receive (in BoutReals)

int ybufflen

Array<BoutReal> umsg_sendbuff

Sending buffers.

Array<BoutReal> dmsg_sendbuff

Array<BoutReal> imsg_sendbuff

Array<BoutReal> omsg_sendbuff

Array<BoutReal> umsg_recvbuff

Receiving buffers.

Array<BoutReal> dmsg_recvbuff

Array<BoutReal> imsg_recvbuff

Array<BoutReal> omsg_recvbuff

bool in_progress

Is the communication still going?

bool include_x_corners

Are corner cells included in x-communication?

bool has_y_communication

Is there a y-communication.

FieldGroup var_list

List of fields being communicated.

struct ConnectionInfo

#include <boutmesh.hxx>

Communication parameters calculated by topology.

Public Members

bool TS_up_in

bool TS_up_out

bool TS_down_in

bool TS_down_out

int UDATA_INDEST

int UDATA_OUTDEST

int UDATA_XSPLIT

int DDATA_INDEST

int DDATA_OUTDEST

int DDATA_XSPLIT

int IDATA_DEST

int ODATA_DEST

struct XDecompositionIndices

Public Members

int ixseps1

int ixseps2

struct YDecompositionIndices

#include <boutmesh.hxx>

Structure for setYDecompositionIndices input/output values.

Public Members

int jyseps1_1

int jyseps2_1

int jyseps1_2

int jyseps2_2

int ny_inner

namespace bout

Provides access to the Hypre library, handling initialisation and finalisation.

Usage

#include <bout/hyprelib.hxx>

class MyClass { public:

private: HypreLib lib; };

This will then automatically initialise Hypre the first time an object is created, and finalise it when the last object is destroyed.

Copyright 2012 B.D.Dudson, S.Farley, M.V.Umansky, X.Q.Xu

Contact: Ben Dudson, bd512@york.ac.uk

This file is part of BOUT++.

BOUT++ is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

BOUT++ is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with BOUT++. If not, see http://www.gnu.org/licenses/.

Information about the version of BOUT++

The build system will update this file on every commit, which may result in files that include it getting rebuilt. Therefore it should be included in as few places as possible

Information about the version of BOUT++

The build system will update this file at configure-time

Explicit inversion of a 3x3 matrix a

If the matrix is singular (ill conditioned), the determinant is return. Otherwise, an empty std::optional is return

SNB model

struct CheckMeshResult

#include <boutmesh.hxx>

Result of bout::checkMesh: bool + reason for failure

This would be nicer as std::optional

Public Members

bool success

std::string reason