UCX  1.8
Unified Communication X

Data Structures

struct  uct_tag_context
 Posted tag context. More...
 

Typedefs

typedef ucs_status_t(* uct_tag_unexp_eager_cb_t) (void *arg, void *data, size_t length, unsigned flags, uct_tag_t stag, uint64_t imm, void **context)
 Callback to process unexpected eager tagged message. More...
 
typedef ucs_status_t(* uct_tag_unexp_rndv_cb_t) (void *arg, unsigned flags, uint64_t stag, const void *header, unsigned header_length, uint64_t remote_addr, size_t length, const void *rkey_buf)
 Callback to process unexpected rendezvous tagged message. More...
 

Functions

ucs_status_t uct_ep_tag_eager_short (uct_ep_h ep, uct_tag_t tag, const void *data, size_t length)
 Short eager tagged-send operation. More...
 
ssize_t uct_ep_tag_eager_bcopy (uct_ep_h ep, uct_tag_t tag, uint64_t imm, uct_pack_callback_t pack_cb, void *arg, unsigned flags)
 Bcopy eager tagged-send operation. More...
 
ucs_status_t uct_ep_tag_eager_zcopy (uct_ep_h ep, uct_tag_t tag, uint64_t imm, const uct_iov_t *iov, size_t iovcnt, unsigned flags, uct_completion_t *comp)
 Zcopy eager tagged-send operation. More...
 
ucs_status_ptr_t uct_ep_tag_rndv_zcopy (uct_ep_h ep, uct_tag_t tag, const void *header, unsigned header_length, const uct_iov_t *iov, size_t iovcnt, unsigned flags, uct_completion_t *comp)
 Rendezvous tagged-send operation. More...
 
ucs_status_t uct_ep_tag_rndv_cancel (uct_ep_h ep, void *op)
 Cancel outstanding rendezvous operation. More...
 
ucs_status_t uct_ep_tag_rndv_request (uct_ep_h ep, uct_tag_t tag, const void *header, unsigned header_length, unsigned flags)
 Send software rendezvous request. More...
 
ucs_status_t uct_iface_tag_recv_zcopy (uct_iface_h iface, uct_tag_t tag, uct_tag_t tag_mask, const uct_iov_t *iov, size_t iovcnt, uct_tag_context_t *ctx)
 Post a tag to a transport interface. More...
 
ucs_status_t uct_iface_tag_recv_cancel (uct_iface_h iface, uct_tag_context_t *ctx, int force)
 Cancel a posted tag. More...
 

Detailed Description

Defines tag matching operations.

Typedef Documentation

◆ uct_tag_unexp_eager_cb_t

typedef ucs_status_t(* uct_tag_unexp_eager_cb_t) (void *arg, void *data, size_t length, unsigned flags, uct_tag_t stag, uint64_t imm, void **context)

This callback is invoked when tagged message sent by eager protocol has arrived and no corresponding tag has been posted.

Note
The callback is always invoked from the context (thread, process) that called uct_iface_progress().
It is allowed to call other communication routines from the callback.
Parameters
[in]argUser-defined argument
[in]dataPoints to the received unexpected data.
[in]lengthLength of data.
[in]flagsMask with uct_cb_param_flags flags. If it contains UCT_CB_PARAM_FLAG_DESC value, this means data is part of a descriptor which must be released later using uct_iface_release_desc by the user if the callback returns UCS_INPROGRESS.
[in]stagTag from sender.
[in]immImmediate data from sender.
[in,out]contextStorage for a per-message user-defined context. In this context, the message is defined by the sender side as a single call to uct_ep_tag_eager_short/bcopy/zcopy. On the transport level the message can be fragmented and delivered to the target over multiple fragments. The fragments will preserve the original order of the message. Each fragment will result in invocation of the above callback. The user can use UCT_CB_PARAM_FLAG_FIRST to identify the first fragment, allocate the context object and use the context as a token that is set by the user and passed to subsequent callbacks of the same message. The user is responsible for allocation and release of the context.
Note
No need to allocate the context in the case of a single fragment message (i.e. flags contains UCT_CB_PARAM_FLAG_FIRST, but does not contain UCT_CB_PARAM_FLAG_MORE).
Return values
UCS_OK- data descriptor was consumed, and can be released by the caller.
UCS_INPROGRESS- data descriptor is owned by the callee, and will be released later.

◆ uct_tag_unexp_rndv_cb_t

typedef ucs_status_t(* uct_tag_unexp_rndv_cb_t) (void *arg, unsigned flags, uint64_t stag, const void *header, unsigned header_length, uint64_t remote_addr, size_t length, const void *rkey_buf)

This callback is invoked when rendezvous send notification has arrived and no corresponding tag has been posted.

Note
The callback is always invoked from the context (thread, process) that called uct_iface_progress().
It is allowed to call other communication routines from the callback.
Parameters
[in]argUser-defined argument
[in]flagsMask with uct_cb_param_flags
[in]stagTag from sender.
[in]headerUser defined header.
[in]header_lengthUser defined header length in bytes.
[in]remote_addrSender's buffer virtual address.
[in]lengthSender's buffer length.
[in]rkey_bufSender's buffer packed remote key. It can be passed to uct_rkey_unpack() to create uct_rkey_t.
Warning
If the user became the owner of the desc (by returning UCS_INPROGRESS) the descriptor must be released later by uct_iface_release_desc by the user.
Return values
UCS_OK- descriptor was consumed, and can be released by the caller.
UCS_INPROGRESS- descriptor is owned by the callee, and would be released later.

Function Documentation

◆ uct_ep_tag_eager_short()

ucs_status_t uct_ep_tag_eager_short ( uct_ep_h  ep,
uct_tag_t  tag,
const void *  data,
size_t  length 
)

This routine sends a message using short eager protocol. Eager protocol means that the whole data is sent to the peer immediately without any preceding notification. The data is provided as buffer and its length,and must not be larger than the corresponding max_short value in uct_iface_attr. The immediate value delivered to the receiver is implicitly equal to 0. If it's required to pass nonzero imm value, uct_ep_tag_eager_bcopy should be used.

Parameters
[in]epDestination endpoint handle.
[in]tagTag to use for the eager message.
[in]dataData to send.
[in]lengthData length.
Returns
UCS_OK - operation completed successfully.
UCS_ERR_NO_RESOURCE - could not start the operation due to lack of send resources.

◆ uct_ep_tag_eager_bcopy()

ssize_t uct_ep_tag_eager_bcopy ( uct_ep_h  ep,
uct_tag_t  tag,
uint64_t  imm,
uct_pack_callback_t  pack_cb,
void *  arg,
unsigned  flags 
)

This routine sends a message using bcopy eager protocol. Eager protocol means that the whole data is sent to the peer immediately without any preceding notification. Custom data callback is used to copy the data to the network buffers.

Note
The resulted data length must not be larger than the corresponding max_bcopy value in uct_iface_attr.
Parameters
[in]epDestination endpoint handle.
[in]tagTag to use for the eager message.
[in]immImmediate value which will be available to the receiver.
[in]pack_cbUser callback to pack the data.
[in]argCustom argument to pack_cb.
[in]flagsTag message flags, see uct_msg_flags.
Returns
>=0 - The size of the data packed by pack_cb.
otherwise - Error code.

◆ uct_ep_tag_eager_zcopy()

ucs_status_t uct_ep_tag_eager_zcopy ( uct_ep_h  ep,
uct_tag_t  tag,
uint64_t  imm,
const uct_iov_t iov,
size_t  iovcnt,
unsigned  flags,
uct_completion_t comp 
)

This routine sends a message using zcopy eager protocol. Eager protocol means that the whole data is sent to the peer immediately without any preceding notification. The input data (which has to be previously registered) in iov array of uct_iov_t structures sent to remote side ("gather output"). Buffers in iov are processed in array order, so the function complete iov[0] before proceeding to iov[1], and so on.

Note
The resulted data length must not be larger than the corresponding max_zcopy value in uct_iface_attr.
Parameters
[in]epDestination endpoint handle.
[in]tagTag to use for the eager message.
[in]immImmediate value which will be available to the receiver.
[in]iovPoints to an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required.
[in]iovcntSize of the iov array. If iovcnt is zero, the data is considered empty. Note that iovcnt is limited by the corresponding max_iov value in uct_iface_attr.
[in]flagsTag message flags, see uct_msg_flags.
[in]compCompletion callback which will be called when the data is reliably received by the peer, and the buffer can be reused or invalidated.
Returns
UCS_OK - operation completed successfully.
UCS_ERR_NO_RESOURCE - could not start the operation due to lack of send resources.
UCS_INPROGRESS - operation started, and comp will be used to notify when it's completed.

◆ uct_ep_tag_rndv_zcopy()

ucs_status_ptr_t uct_ep_tag_rndv_zcopy ( uct_ep_h  ep,
uct_tag_t  tag,
const void *  header,
unsigned  header_length,
const uct_iov_t iov,
size_t  iovcnt,
unsigned  flags,
uct_completion_t comp 
)

This routine sends a message using rendezvous protocol. Rendezvous protocol means that only a small notification is sent at first, and the data itself is transferred later (when there is a match) to avoid extra memory copy.

Note
The header will be available to the receiver in case of unexpected rendezvous operation only, i.e. the peer has not posted tag for this message yet (by means of uct_iface_tag_recv_zcopy), when it is arrived.
Parameters
[in]epDestination endpoint handle.
[in]tagTag to use for the eager message.
[in]headerUser defined header.
[in]header_lengthUser defined header length in bytes. Note that it is limited by the corresponding max_hdr value in uct_iface_attr.
[in]iovPoints to an array of uct_iov_t structures. A particular structure pointer must be valid address. A NULL terminated array is not required.
[in]iovcntSize of the iov array. If iovcnt is zero, the data is considered empty. Note that iovcnt is limited by the corresponding max_iov value in uct_iface_attr.
[in]flagsTag message flags, see uct_msg_flags.
[in]compCompletion callback which will be called when the data is reliably received by the peer, and the buffer can be reused or invalidated.
Returns
>=0 - The operation is in progress and the return value is a handle which can be used to cancel the outstanding rendezvous operation.
otherwise - Error code.

◆ uct_ep_tag_rndv_cancel()

ucs_status_t uct_ep_tag_rndv_cancel ( uct_ep_h  ep,
void *  op 
)

This routine signals the underlying transport disregard the outstanding operation without calling completion callback provided in uct_ep_tag_rndv_zcopy.

Note
The operation handle should be valid at the time the routine is invoked. I.e. it should be a handle of the real operation which is not completed yet.
Parameters
[in]epDestination endpoint handle.
[in]opRendezvous operation handle, as returned from uct_ep_tag_rndv_zcopy.
Returns
UCS_OK - The operation has been canceled.

◆ uct_ep_tag_rndv_request()

ucs_status_t uct_ep_tag_rndv_request ( uct_ep_h  ep,
uct_tag_t  tag,
const void *  header,
unsigned  header_length,
unsigned  flags 
)

This routine sends a rendezvous request only, which indicates that the data transfer should be completed in software.

Parameters
[in]epDestination endpoint handle.
[in]tagTag to use for matching.
[in]headerUser defined header
[in]header_lengthUser defined header length in bytes. Note that it is limited by the corresponding max_hdr value in uct_iface_attr.
[in]flagsTag message flags, see uct_msg_flags.
Returns
UCS_OK - operation completed successfully.
UCS_ERR_NO_RESOURCE - could not start the operation due to lack of send resources.

◆ uct_iface_tag_recv_zcopy()

ucs_status_t uct_iface_tag_recv_zcopy ( uct_iface_h  iface,
uct_tag_t  tag,
uct_tag_t  tag_mask,
const uct_iov_t iov,
size_t  iovcnt,
uct_tag_context_t ctx 
)

This routine posts a tag to be matched on a transport interface. When a message with the corresponding tag arrives it is stored in the user buffer (described by iov and iovcnt) directly. The operation completion is reported using callbacks on the ctx structure.

Parameters
[in]ifaceInterface to post the tag on.
[in]tagTag to expect.
[in]tag_maskMask which specifies what bits of the tag to compare.
[in]iovPoints to an array of uct_iov_t structures. The iov pointer must be a valid address of an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required.
[in]iovcntSize of the iov data uct_iov_t structures array. If iovcnt is zero, the data is considered empty. iovcnt is limited by uct_iface_attr::cap::tag::max_iov.
[in,out]ctxContext associated with this particular tag, "priv" field in this structure is used to track the state internally.
Returns
UCS_OK - The tag is posted to the transport.
UCS_ERR_NO_RESOURCE - Could not start the operation due to lack of resources.
UCS_ERR_EXCEEDS_LIMIT - No more room for tags in the transport.

◆ uct_iface_tag_recv_cancel()

ucs_status_t uct_iface_tag_recv_cancel ( uct_iface_h  iface,
uct_tag_context_t ctx,
int  force 
)

This routine cancels a tag, which was previously posted by uct_iface_tag_recv_zcopy. The tag would be either matched or canceled, in a bounded time, regardless of the peer actions. The original completion callback of the tag would be called with the status if force is not set.

Parameters
[in]ifaceInterface to cancel the tag on.
[in]ctxTag context which was used for posting the tag. If force is 0, ctx->completed_cb will be called with either UCS_OK which means the tag was matched and data received despite the cancel request, or UCS_ERR_CANCELED which means the tag was successfully canceled before it was matched.
[in]forceWhether to report completions to ctx->completed_cb. If nonzero, the cancel is assumed to be successful, and the callback is not called.
Returns
UCS_OK - The tag is canceled in the transport.