Manages MPI Communication with a set of known communication partners.
Features / Restrictions:
When running multiple BufferSystems concurrently different MPI tags have to be used for the systems: the tag can be passed in the constructor.
#include <BufferSystem.h>
Classes | |
class | iterator |
struct | SendInfo |
Public Member Functions | |
Constructors | |
GenericBufferSystem (const MPI_Comm &communicator, int tag=0) | |
GenericBufferSystem (const GenericBufferSystem &other) | |
GenericBufferSystem & | operator= (const GenericBufferSystem &other) |
~GenericBufferSystem ()=default | |
Receiver Registration | |
| |
template<typename RankIter > | |
void | setReceiverInfo (RankIter begin, RankIter end, bool changingSize) |
template<typename Range > | |
void | setReceiverInfo (const Range &range, bool changingSize) |
void | setReceiverInfo (const std::set< MPIRank > &ranksToRecvFrom, bool changingSize) |
Sets receiver information, when message sizes are unknown. More... | |
void | setReceiverInfo (const std::map< MPIRank, MPISize > &ranksToRecvFrom) |
Sets receiver information, when message sizes are known. More... | |
void | setReceiverInfoFromSendBufferState (bool useSizeFromSendBuffers, bool changingSize) |
Sets receiver information, using SendBuffers (symmetric communication) More... | |
void | sizeHasChanged (bool alwaysChangingSize=false) |
Notifies that GenericBufferSystem that message sizes have changed ( and optionally are changing in all following steps) More... | |
Executing Communication Step | |
void | scheduleReceives () |
SendBuffer_T & | sendBuffer (MPIRank rank) |
Returns an existing SendBuffer, or creates a new one (only if !isCommunicationRunning() ) More... | |
SendBuffer_T & | sendBuffer (uint_t rank) |
size_t | size () const |
void | sendAll () |
Sends filled ( nonzero length) SendBuffers to their destination ranks. More... | |
void | send (MPIRank rank) |
Sends a single SendBuffer to its destination rank. More... | |
iterator | begin () |
iterator | end () |
Status Queries | |
| |
bool | isSizeCommunicatedInNextStep () const |
bool | isCommunicationRunning () const |
bool | isReceiverInformationSet () const |
void | useIProbe (const bool use) |
bool | isIProbedUsed () const |
int64_t | getBytesSent () const |
Bytes sent during the current or last communication. More... | |
int64_t | getBytesReceived () const |
Bytes received during the current or last communication. More... | |
int64_t | getNumberOfSends () const |
Communication partners during current or last send operation. More... | |
int64_t | getNumberOfReceives () const |
Communication partners during current or last receive operation. More... | |
Friends | |
Iterator | |
| |
class | iterator |
Rank Ranges | |
using | RankRange = std::set< MPIRank > |
class | GenericOpenMPBufferSystem< RecvBuffer_T, SendBuffer_T > |
internal::KnownSizeCommunication< RecvBuffer_T, SendBuffer_T > | knownSizeComm_ |
internal::UnknownSizeCommunication< RecvBuffer_T, SendBuffer_T > | unknownSizeComm_ |
internal::UnknownSizeCommunicationIProbe< RecvBuffer_T, SendBuffer_T > | unknownSizeCommIProbe_ |
internal::NoMPICommunication< RecvBuffer_T, SendBuffer_T > | noMPIComm_ |
internal::AbstractCommunication< RecvBuffer_T, SendBuffer_T > * | currentComm_ |
bool | sizeChangesEverytime_ |
bool | communicationRunning_ |
std::map< MPIRank, typename internal::AbstractCommunication< RecvBuffer_T, SendBuffer_T >::ReceiveInfo > | recvInfos_ |
Info about the message to be received from a certain rank: information holds the buffer and, if known, the message size. More... | |
std::map< MPIRank, SendInfo > | sendInfos_ |
bool | useIProbe_ = false |
switch between IProbe and two message communication for varying size communication More... | |
int64_t | bytesSent_ = 0 |
number of bytes sent during last communication More... | |
int64_t | bytesReceived_ = 0 |
number of bytes received during last communication More... | |
int64_t | numberOfSends_ = 0 |
number of communication partners during last send More... | |
int64_t | numberOfReceives_ = 0 |
number of communication partners during last receive More... | |
static std::set< int > | activeTags_ |
static RankRange | noRanks () |
static RankRange | allRanks () |
static RankRange | allRanksButRoot () |
static RankRange | onlyRoot () |
static RankRange | onlyRank (MPIRank includedRank) |
void | startCommunication () |
Starts communication. More... | |
void | endCommunication () |
Cleanup after communication. More... | |
RecvBuffer_T * | waitForNext (MPIRank &fromRank) |
Helper function for iterator. More... | |
void | setCommunicationType (const bool knownSize) |
Sets the communication type to known size, unknown size or NoMPI comm. More... | |
using walberla::mpi::GenericBufferSystem< RecvBuffer_T, SendBuffer_T >::RankRange = std::set<MPIRank> |
|
explicit |
walberla::mpi::GenericBufferSystem< Rb, Sb >::GenericBufferSystem | ( | const GenericBufferSystem< RecvBuffer_T, SendBuffer_T > & | other | ) |
|
default |
|
static |
|
static |
|
inline |
|
inline |
|
protected |
Cleanup after communication.
|
inline |
Bytes received during the current or last communication.
|
inline |
Bytes sent during the current or last communication.
|
inline |
Communication partners during current or last receive operation.
|
inline |
Communication partners during current or last send operation.
|
inline |
|
inline |
|
inline |
|
inline |
|
static |
|
static |
|
static |
GenericBufferSystem< Rb, Sb > & walberla::mpi::GenericBufferSystem< Rb, Sb >::operator= | ( | const GenericBufferSystem< RecvBuffer_T, SendBuffer_T > & | other | ) |
|
inline |
void walberla::mpi::GenericBufferSystem< Rb, Sb >::send | ( | MPIRank | rank | ) |
Sends a single SendBuffer to its destination rank.
If SendBuffer is empty no message is sent.
If communication was not started before, it is started with this function.
void walberla::mpi::GenericBufferSystem< Rb, Sb >::sendAll |
Sends filled ( nonzero length) SendBuffers to their destination ranks.
If some of the SendBuffers have been sent manually before using send(int rank) they are skipped, only the remaining buffers are sent.
If communication was not started before, it is started with this function.
Sb & walberla::mpi::GenericBufferSystem< Rb, Sb >::sendBuffer | ( | MPIRank | rank | ) |
Returns an existing SendBuffer, or creates a new one (only if !isCommunicationRunning() )
rank | the rank where the buffer should be sent to |
|
inline |
|
protected |
Sets the communication type to known size, unknown size or NoMPI comm.
void walberla::mpi::GenericBufferSystem< Rb, Sb >::setReceiverInfo | ( | const Range & | range, |
bool | changingSize | ||
) |
void walberla::mpi::GenericBufferSystem< Rb, Sb >::setReceiverInfo | ( | const std::map< MPIRank, MPISize > & | ranksToRecvFrom | ) |
Sets receiver information, when message sizes are known.
ranksToRecvFrom | Map containing all ranks, where messages are received from, as keys and the message sizes as values. The message sizes are expected to be constant for all communication step until behavior is changed with setReceiverInfo*() or sizeHasChanged() |
void walberla::mpi::GenericBufferSystem< Rb, Sb >::setReceiverInfo | ( | const std::set< MPIRank > & | ranksToRecvFrom, |
bool | changingSize | ||
) |
Sets receiver information, when message sizes are unknown.
ranksToRecvFrom | set of all ranks where messages are received |
changingSize | true if the message size is different in each communication step. If false the message size is exchanged once and is expected to be constant If true the message size is exchanged before each communication step. The behavior can be changed later one using setReceiverInfo() or sizeHasChanged(). |
void walberla::mpi::GenericBufferSystem< Rb, Sb >::setReceiverInfo | ( | RankIter | begin, |
RankIter | end, | ||
bool | changingSize | ||
) |
void walberla::mpi::GenericBufferSystem< Rb, Sb >::setReceiverInfoFromSendBufferState | ( | bool | useSizeFromSendBuffers, |
bool | changingSize | ||
) |
Sets receiver information, using SendBuffers (symmetric communication)
Gives the GenericBufferSystem the information that messages are received from the same processes that we send to (i.e. from all ranks where SendBuffers were already filled ) sendBuffer() has to be called before, and corresponding SendBuffers have to be filled.
useSizeFromSendBuffers | If true, the sizes are expected to be known and equal to the size of the SendBuffers. SendBuffers with zero size are ignored. If false, all SendBuffers (also with zero size) are registered as ranks where messages are received from. The size is unknown, and communicated before. |
changingSize | if true the size is communicated before every communication step. if useSizeFromSendBuffer==true and changingSize==true, the size is not communicated in the first step but in all following steps. |
|
inline |
void walberla::mpi::GenericBufferSystem< Rb, Sb >::sizeHasChanged | ( | bool | alwaysChangingSize = false | ) |
Notifies that GenericBufferSystem that message sizes have changed ( and optionally are changing in all following steps)
Useful when setReceiverInfo was set such that GenericBufferSystem assumes constant message sizes for all steps. Can only be called if no communication is currently running.
alwaysChangingSize | if true the message sizes is communicated in all following steps, if false only in the next step. |
|
protected |
Starts communication.
|
inline |
|
protected |
Helper function for iterator.
See documentation of AbstractCommunication::waitForNext()
|
friend |
|
friend |
|
staticprotected |
|
protected |
number of bytes received during last communication
|
protected |
number of bytes sent during last communication
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
number of communication partners during last receive
|
protected |
number of communication partners during last send
|
protected |
Info about the message to be received from a certain rank: information holds the buffer and, if known, the message size.
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
switch between IProbe and two message communication for varying size communication