UCP Endpoint

Data Structures
struct	ucp_stream_poll_ep
	Output parameter of ucp_stream_worker_poll function. More...
struct	ucp_ep_params
	Tuning parameters for the UCP endpoint. More...

Typedefs
typedef struct ucp_stream_poll_ep	ucp_stream_poll_ep_t
	Output parameter of ucp_stream_worker_poll function. More...
typedef struct ucp_ep *	ucp_ep_h
	UCP Endpoint. More...
typedef struct ucp_conn_request *	ucp_conn_request_h
	UCP connection request. More...
typedef ucs_status_t(*	ucp_am_callback_t) (void *arg, void *data, size_t length, ucp_ep_h reply_ep, unsigned flags)
	Callback to process incoming Active Message. More...
typedef struct ucp_ep_params	ucp_ep_params_t
	Tuning parameters for the UCP endpoint. More...

Enumerations
enum	ucp_ep_params_field {   UCP_EP_PARAM_FIELD_REMOTE_ADDRESS = UCS_BIT(0), UCP_EP_PARAM_FIELD_ERR_HANDLING_MODE = UCS_BIT(1), UCP_EP_PARAM_FIELD_ERR_HANDLER = UCS_BIT(2), UCP_EP_PARAM_FIELD_USER_DATA = UCS_BIT(3),   UCP_EP_PARAM_FIELD_SOCK_ADDR = UCS_BIT(4), UCP_EP_PARAM_FIELD_FLAGS = UCS_BIT(5), UCP_EP_PARAM_FIELD_CONN_REQUEST = UCS_BIT(6) }
	UCP endpoint parameters field mask. More...
enum	ucp_ep_params_flags_field { UCP_EP_PARAMS_FLAGS_CLIENT_SERVER = UCS_BIT(0), UCP_EP_PARAMS_FLAGS_NO_LOOPBACK = UCS_BIT(1) }
	UCP endpoint parameters flags. More...
enum	ucp_ep_close_mode { UCP_EP_CLOSE_MODE_FORCE = 0, UCP_EP_CLOSE_MODE_FLUSH = 1 }
	Close UCP endpoint modes. More...
enum	ucp_cb_param_flags { UCP_CB_PARAM_FLAG_DATA = UCS_BIT(0) }
	Descriptor flags for Active Message callback. More...
enum	ucp_err_handling_mode_t { UCP_ERR_HANDLING_MODE_NONE, UCP_ERR_HANDLING_MODE_PEER }
	Error handling mode for the UCP endpoint. More...

Functions
ucs_status_t	ucp_ep_create (ucp_worker_h worker, const ucp_ep_params_t *params, ucp_ep_h *ep_p)
	Create and connect an endpoint. More...
ucs_status_ptr_t	ucp_ep_close_nb (ucp_ep_h ep, unsigned mode)
	Non-blocking endpoint closure. More...
void	ucp_ep_print_info (ucp_ep_h ep, FILE *stream)
	Print endpoint information. More...
ucs_status_ptr_t	ucp_ep_flush_nb (ucp_ep_h ep, unsigned flags, ucp_send_callback_t cb)
	Non-blocking flush of outstanding AMO and RMA operations on the endpoint. More...
void	ucp_request_release (void *request)
void	ucp_ep_destroy (ucp_ep_h ep)
ucs_status_ptr_t	ucp_disconnect_nb (ucp_ep_h ep)
ucs_status_t	ucp_request_test (void *request, ucp_tag_recv_info_t *info)
ucs_status_t	ucp_ep_flush (ucp_ep_h ep)
ucs_status_ptr_t	ucp_ep_modify_nb (ucp_ep_h ep, const ucp_ep_params_t *params)
	Modify endpoint parameters. More...

Detailed Description

UCP Endpoint routines

---

Data Structure Documentation

ucp_stream_poll_ep

struct ucp_stream_poll_ep

The structure defines the endpoint and its user data.

Data Fields
ucp_ep_h	ep	Endpoint handle.
void *	user_data	User data associated with an endpoint passed in ucp_ep_params_t::user_data.
unsigned	flags	Reserved for future use.
uint8_t	reserved[16]	Reserved for future use.

ucp_ep_params

struct ucp_ep_params

The structure defines the parameters that are used for the UCP endpoint tuning during the UCP ep creation.

Examples

ucp_hello_world.c.

Data Fields
uint64_t	field_mask	Mask of valid fields in this structure, using bits from ucp_ep_params_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields.
const ucp_address_t *	address	Destination address; this field should be set along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_REMOTE_ADDRESS and must be obtained using ucp_worker_get_address.
ucp_err_handling_mode_t	err_mode	Desired error handling mode, optional parameter. Default value is UCP_ERR_HANDLING_MODE_NONE.
ucp_err_handler_t	err_handler	Handler to process transport level failure.
void *	user_data	User data associated with an endpoint. See ucp_stream_poll_ep_t and ucp_err_handler_t
unsigned	flags	Endpoint flags from ucp_ep_params_flags_field. This value is optional. If it's not set (along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_FLAGS), the ucp_ep_create() routine will consider the flags as set to zero.
ucs_sock_addr_t	sockaddr	Destination address in the form of a sockaddr; this field should be set along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_SOCK_ADDR and must be obtained from the user, it means that this type of the endpoint creation is possible only on client side in client-server connection establishment flow.
ucp_conn_request_h	conn_request	Connection request from client; this field should be set along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_CONN_REQUEST and must be obtained from ucp_listener_conn_callback_t, it means that this type of the endpoint creation is possible only on server side in client-server connection establishment flow.

Typedef Documentation

ucp_stream_poll_ep_t

typedef struct ucp_stream_poll_ep ucp_stream_poll_ep_t

The structure defines the endpoint and its user data.

ucp_ep_h

typedef struct ucp_ep* ucp_ep_h

The endpoint handle is an opaque object that is used to address a remote worker. It typically provides a description of source, destination, or both. All UCP communication routines address a destination with the endpoint handle. The endpoint handle is associated with only one UCP context. UCP provides the endpoint create routine to create the endpoint handle and the destroy routine to destroy the endpoint handle.

ucp_conn_request_h

typedef struct ucp_conn_request* ucp_conn_request_h

A server-side handle to incoming connection request. Can be used to create an endpoint which connects back to the client.

ucp_am_callback_t

typedef ucs_status_t(* ucp_am_callback_t) (void *arg, void *data, size_t length, ucp_ep_h reply_ep, unsigned flags)

When the callback is called, flags indicates how data should be handled.

Parameters

[in]	arg	User-defined argument.
[in]	data	Points to the received data. This data may persist after the callback returns and needs to be freed with ucp_am_data_release.
[in]	length	Length of data.
[in]	reply_ep	If the Active Message is sent with the UCP_AM_SEND_REPLY flag, the sending ep will be passed in. If not, NULL will be passed.
[in]	flags	If this flag is set to UCP_CB_PARAM_FLAG_DATA, the callback can return UCS_INPROGRESS and data will persist after the callback returns.

Returns

UCS_OK data will not persist after the callback returns.

UCS_INPROGRESS Can only be returned if flags is set to UCP_CB_PARAM_FLAG_DATA. If UCP_INPROGRESS is returned, data will persist after the callback has returned. To free the memory, a pointer to the data must be passed into ucp_am_data_release.

Note

This callback should be set and released by ucp_worker_set_am_handler function.

ucp_ep_params_t

typedef struct ucp_ep_params ucp_ep_params_t

The structure defines the parameters that are used for the UCP endpoint tuning during the UCP ep creation.

Enumeration Type Documentation

ucp_ep_params_field

enum ucp_ep_params_field

The enumeration allows specifying which fields in ucp_ep_params_t are present. It is used to enable backward compatibility support.

Enumerator
UCP_EP_PARAM_FIELD_REMOTE_ADDRESS	Address of remote peer
UCP_EP_PARAM_FIELD_ERR_HANDLING_MODE	Error handling mode. ucp_err_handling_mode_t
UCP_EP_PARAM_FIELD_ERR_HANDLER	Handler to process transport level errors
UCP_EP_PARAM_FIELD_USER_DATA	User data pointer
UCP_EP_PARAM_FIELD_SOCK_ADDR	Socket address field
UCP_EP_PARAM_FIELD_FLAGS	Endpoint flags
UCP_EP_PARAM_FIELD_CONN_REQUEST	Connection request field

ucp_ep_params_flags_field

enum ucp_ep_params_flags_field

The enumeration list describes the endpoint's parameters flags supported by ucp_ep_create() function.

Enumerator
UCP_EP_PARAMS_FLAGS_CLIENT_SERVER	Using a client-server connection establishment mechanism. ucs_sock_addr_t sockaddr field must be provided and contain the address of the remote peer
UCP_EP_PARAMS_FLAGS_NO_LOOPBACK	Avoid connecting the endpoint to itself when connecting the endpoint to the same worker it was created on. Affects protocols which send to a particular remote endpoint, for example stream

ucp_ep_close_mode

enum ucp_ep_close_mode

The enumeration is used to specify the behavior of ucp_ep_close_nb.

Enumerator
UCP_EP_CLOSE_MODE_FORCE	ucp_ep_close_nb releases the endpoint without any confirmation from the peer. All outstanding requests will be completed with UCS_ERR_CANCELED error. Note This mode may cause transport level errors on remote side, so it requires set UCP_ERR_HANDLING_MODE_PEER for all endpoints created on both (local and remote) sides to avoid undefined behavior.
UCP_EP_CLOSE_MODE_FLUSH	ucp_ep_close_nb schedules flushes on all outstanding operations.

ucp_cb_param_flags

enum ucp_cb_param_flags

In a callback, if flags is set to UCP_CB_PARAM_FLAG_DATA in a callback then data was allocated, so if UCS_INPROGRESS is returned from the callback, the data parameter will persist and the user has to call ucp_am_data_release when data is no longer needed.

Enumerator
UCP_CB_PARAM_FLAG_DATA

ucp_err_handling_mode_t

enum ucp_err_handling_mode_t

Specifies error handling mode for the UCP endpoint.

Enumerator
UCP_ERR_HANDLING_MODE_NONE	No guarantees about error reporting, imposes minimal overhead from a performance perspective. Note In this mode, any error reporting will not generate calls to ucp_ep_params_t::err_handler.
UCP_ERR_HANDLING_MODE_PEER	Guarantees that send requests are always completed (successfully or error) even in case of remote failure, disables protocols and APIs which may cause a hang or undefined behavior in case of peer failure, may affect performance and memory footprint

Function Documentation

ucp_ep_create()

ucs_status_t ucp_ep_create	(	ucp_worker_h	worker,
		const ucp_ep_params_t *	params,
		ucp_ep_h *	ep_p
	)

This routine creates and connects an endpoint on a local worker for a destination address that identifies the remote worker. This function is non-blocking, and communications may begin immediately after it returns. If the connection process is not completed, communications may be delayed. The created endpoint is associated with one and only one worker.

Parameters

[in]	worker	Handle to the worker; the endpoint is associated with the worker.
[in]	params	User defined ucp_ep_params_t configurations for the UCP endpoint.
[out]	ep_p	A handle to the created endpoint.

Returns

Error code as defined by ucs_status_t

Note

One of the following fields has to be specified:

• ucp_ep_params_t::address
• ucp_ep_params_t::sockaddr
• ucp_ep_params_t::conn_request

By default, ucp_ep_create() will connect an endpoint to itself if the endpoint is destined to the same worker on which it was created, i.e. params.address belongs to worker. This behavior can be changed by passing the UCP_EP_PARAMS_FLAGS_NO_LOOPBACK flag in params.flags. In that case, the endpoint will be connected to the next endpoint created in the same way on the same worker.

Examples

ucp_hello_world.c.

ucp_ep_close_nb()

ucs_status_ptr_t ucp_ep_close_nb	(	ucp_ep_h	ep,
		unsigned	mode
	)

This routine releases the endpoint. The endpoint closure process depends on the selected mode.

Parameters

[in]	ep	Handle to the endpoint to close.
[in]	mode	One from ucp_ep_close_mode value.

Returns

UCS_OK - The endpoint is closed successfully.

UCS_PTR_IS_ERR(_ptr) - The closure failed and an error code indicates the transport level status. However, resources are released and the endpoint can no longer be used.

otherwise - The closure process is started, and can be completed at any point in time. A request handle is returned to the application in order to track progress of the endpoint closure. The application is responsible for releasing the handle using the ucp_request_free routine.

Note

ucp_ep_close_nb replaces deprecated ucp_disconnect_nb and ucp_ep_destroy

ucp_ep_print_info()

void ucp_ep_print_info	(	ucp_ep_h	ep,
		FILE *	stream
	)

This routine prints information about the endpoint transport methods, their thresholds, and other useful information associated with the endpoint.

Parameters

[in]	ep	Endpoint object whose configuration to print.
[in]	stream	Output stream to print the information to.

ucp_ep_flush_nb()

ucs_status_ptr_t ucp_ep_flush_nb	(	ucp_ep_h	ep,
		unsigned	flags,
		ucp_send_callback_t	cb
	)

This routine flushes all outstanding AMO and RMA communications on the endpoint. All the AMO and RMA operations issued on the ep prior to this call are completed both at the origin and at the target endpoint when this call returns.

Parameters

[in]	ep	UCP endpoint.
[in]	flags	Flags for flush operation. Reserved for future use.
[in]	cb	Callback which will be called when the flush operation completes.

Returns

NULL - The flush operation was completed immediately.

UCS_PTR_IS_ERR(_ptr) - The flush operation failed.

otherwise - Flush operation was scheduled and can be completed in any point in time. The request handle is returned to the application in order to track progress. The application is responsible for releasing the handle using ucp_request_free() routine.

The following example demonstrates how blocking flush can be implemented using non-blocking flush:

void empty_function(void *request, ucs_status_t status)
{
}

ucs_status_t blocking_ep_flush(ucp_ep_h ep, ucp_worker_h worker)
{
    void *request;

    request = ucp_ep_flush_nb(ep, 0, empty_function);
    if (request == NULL) {
        return UCS_OK;
    } else if (UCS_PTR_IS_ERR(request)) {
        return UCS_PTR_STATUS(request);
    } else {
        ucs_status_t status;
        do {
            ucp_worker_progress(worker);
            status = ucp_request_check_status(request);
        } while (status == UCS_INPROGRESS);
        ucp_request_free(request);
        return status;
    }
}

Examples

ucp_hello_world.c.

ucp_request_release()

void ucp_request_release

(

void *

request

)

Deprecated:

Replaced by ucp_request_free.

Examples

ucp_hello_world.c.

ucp_ep_destroy()

void ucp_ep_destroy

(

ucp_ep_h

ep

)

Deprecated:

Replaced by ucp_ep_close_nb.

Examples

ucp_hello_world.c.

ucp_disconnect_nb()

ucs_status_ptr_t ucp_disconnect_nb

(

ucp_ep_h

ep

)

Deprecated:

Replaced by ucp_ep_close_nb.

ucp_request_test()

ucs_status_t ucp_request_test	(	void *	request,
		ucp_tag_recv_info_t *	info
	)

Deprecated:

Replaced by ucp_tag_recv_request_test and ucp_request_check_status depends on use case.

Note

Please use ucp_request_check_status for cases that only need to check the completion status of an outstanding request. ucp_request_check_status can be used for any type of request. ucp_tag_recv_request_test should only be used for requests returned by ucp_tag_recv_nb (or request allocated by user for ucp_tag_recv_nbr) for which additional information (returned via the info pointer) is needed.

ucp_ep_flush()

ucs_status_t ucp_ep_flush

(

ucp_ep_h

ep

)

Deprecated:

Replaced by ucp_ep_flush_nb.

ucp_ep_modify_nb()

ucs_status_ptr_t ucp_ep_modify_nb	(	ucp_ep_h	ep,
		const ucp_ep_params_t *	params
	)

Deprecated:

Use ucp_listener_conn_handler_t instead of ucp_listener_accept_handler_t, if you have other use case please submit an issue on https://github.com/openucx/ucx or report to ucx-g.nosp@m.roup.nosp@m.@elis.nosp@m.t.or.nosp@m.nl.go.nosp@m.v

This routine modifies endpoint created by ucp_ep_create or ucp_listener_accept_callback_t. For example, this API can be used to setup custom parameters like ucp_ep_params_t::user_data or ucp_ep_params_t::err_handler to endpoint created by ucp_listener_accept_callback_t.

Parameters

[in]	ep	A handle to the endpoint.
[in]	params	User defined ucp_ep_params_t configurations for the UCP endpoint.

Returns

NULL - The endpoint is modified successfully.

UCS_PTR_IS_ERR(_ptr) - The reconfiguration failed and an error code indicates the status. However, the endpoint is not modified and can be used further.

otherwise - The reconfiguration process is started, and can be completed at any point in time. A request handle is returned to the application in order to track progress of the endpoint modification. The application is responsible for releasing the handle using the ucp_request_free routine.

Note

See the documentation of ucp_ep_params_t for details, only some of the parameters can be modified.