 |
waLBerla 7.2
|
|
waLBerla 7.2
|
Data packing/unpacking for ghost layer based communication of a GPUField.
Encapsulate information on how to extract data from blocks that should be communicated to neighboring blocks (see packDataImpl()) and how to inject this data in a receiving block (see unpackData()). This involves a host memory buffer and two device-to-host memory copy operations.
A special method exists for communication between two blocks which are allocated on the same process (see communicateLocal()). In this case the data does not have be communicated via a host buffer, but can be sent directly. This involves a single device-to-device memory copy operation.
Data that is packed in direction "dir" at one block is unpacked in direction "stencil::inverseDir[dir]" at the neighboring block. This behavior must be implemented in communicateLocal()!
See MemcpyPackInfo for a more efficient packing/unpacking method where the buffer is stored in device memory rather than in host memory.
| GPUField_T | A fully qualified GPUField. |
#include <GPUPackInfo.h>
Inheritance diagram for walberla::gpu::communication::GPUPackInfo< GPUField_T >:Public Types | |
| using | FieldType = typename GPUField_T::value_type |
Public Member Functions | |
| GPUPackInfo (const BlockDataID &bdId) | |
| GPUPackInfo (const BlockDataID &bdId, const uint_t numberOfGHostLayers) | |
| ~GPUPackInfo () override=default | |
| bool | constantDataExchange () const override |
| Should return true if the amount of data that is packed for a given block in direction "dir" is guaranteed to remain constant over time. | |
| bool | threadsafeReceiving () const override |
| Must return false if calling unpackData and/or communicateLocal is not thread-safe. | |
| void | unpackData (IBlock *receiver, stencil::Direction dir, mpi::RecvBuffer &buffer) override |
| Unpack received Data. | |
| void | communicateLocal (const IBlock *sender, IBlock *receiver, stencil::Direction dir) override |
| Copy data from one local block to another local block. | |
| void | setCommunicationStream (gpuStream_t stream) |
| Public Member Functions inherited from walberla::communication::UniformPackInfo | |
| UniformPackInfo ()=default | |
| virtual | ~UniformPackInfo ()=default |
| void | packData (const IBlock *sender, stencil::Direction dir, mpi::SendBuffer &buffer) const |
| Pack data from a block into a send buffer. | |
| virtual void | beforeStartCommunication () |
| This function is called once before the communication is started by the UniformBufferedScheme. | |
| virtual void | afterStartCommunication () |
| This function is called once after the communication has been started by the UniformBufferedScheme. | |
| virtual void | beforeWait () |
| This function is called once before the UniformBufferedScheme waits for the communication to finish. | |
| virtual void | afterWait () |
| This function is called once after the communication has been finished by the UniformBufferedScheme. | |
Protected Member Functions | |
| void | packDataImpl (const IBlock *sender, stencil::Direction dir, mpi::SendBuffer &outBuffer) const override |
| Pack data from a block into a send buffer. | |
| uint_t | numberOfGhostLayersToCommunicate (const GPUField_T *const field) const |
| Protected Member Functions inherited from walberla::communication::UniformPackInfo | |
Protected Attributes | |
| const BlockDataID | bdId_ |
| bool | communicateAllGhostLayers_ |
| uint_t | numberOfGhostLayers_ |
| bool | copyAsync_ |
| gpuStream_t | communicationStream_ |
| std::map< stencil::Direction, PinnedMemoryBuffer > | pinnedRecvBuffers_ |
| std::map< stencil::Direction, PinnedMemoryBuffer > | pinnedSendBuffers_ |
| Protected Attributes inherited from walberla::communication::UniformPackInfo | |
| std::map< const IBlock *, std::map< stencil::Direction, size_t > > | bufferSize_ |
| using walberla::gpu::communication::GPUPackInfo< GPUField_T >::FieldType = typename GPUField_T::value_type |
|
inline |
|
inline |
|
overridedefault |
|
overridevirtual |
Copy data from one local block to another local block.
Both blocks are allocated on the current process. If NOT thread-safe, threadsafeReceiving must return false!
| sender | id of block where the data should be copied from |
| receiver | id of block where the data should be copied to |
| dir | the direction of the communication ( from sender to receiver ) |
Implements walberla::communication::UniformPackInfo.
|
inlineoverridevirtual |
Should return true if the amount of data that is packed for a given block in direction "dir" is guaranteed to remain constant over time.
False otherwise. If you are not sure what to return, return false! Returning false is always safe. Falsely returning true will lead to errors! However, if the data can be guaranteed to remain constant over time, returning true enables performance optimizations during the communication.
Implements walberla::communication::UniformPackInfo.
|
protected |
|
overrideprotectedvirtual |
Pack data from a block into a send buffer.
Must be thread-safe!
Implementations must reserve extra space and advance the send buffer pointer according to the alignment of the ghost layer data type! The buffer is char-aligned.
| sender | the block whose data should be packed into a buffer |
| dir | pack data for neighbor in this direction |
| buffer | buffer for writing the data into |
Implements walberla::communication::UniformPackInfo.
|
inline |
|
inlineoverridevirtual |
Must return false if calling unpackData and/or communicateLocal is not thread-safe.
True otherwise. If you are not sure what to return, return false! Returning false is always safe. Falsely returning true will most likely lead to errors! However, if both unpackData AND communicateLocal are thread-safe, returning true can lead to performance improvements.
Implements walberla::communication::UniformPackInfo.
|
overridevirtual |
Unpack received Data.
If NOT thread-safe, threadsafeReceiving must return false!
Implementations must reserve extra space and advance the send buffer pointer according to the alignment of the ghost layer data type! The buffer is char-aligned.
| receiver | the block where the unpacked data should be stored into |
| dir | receive data from neighbor in this direction |
| buffer | buffer for reading the data from |
Implements walberla::communication::UniformPackInfo.
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
mutableprotected |
