UCP Wake-up routines

Functions
ucs_status_t	ucp_worker_get_efd (ucp_worker_h worker, int *fd)
	Obtain an event file descriptor for event notification.
ucs_status_t	ucp_worker_wait (ucp_worker_h worker)
	Wait for an event of the worker.
void	ucp_worker_wait_mem (ucp_worker_h worker, void *address)
	Wait for memory update on the address.
ucs_status_t	ucp_worker_arm (ucp_worker_h worker)
	Turn on event notification for the next event.
ucs_status_t	ucp_worker_signal (ucp_worker_h worker)
	Cause an event of the worker.

Detailed Description

UCP Wake-up routines

Function Documentation

ucp_worker_get_efd()

ucs_status_t ucp_worker_get_efd	(	ucp_worker_h	worker,
		int *	fd )

This routine returns a valid file descriptor for polling functions. The file descriptor will get signaled when an event occurs, as part of the wake-up mechanism. Signaling means a call to poll() or select() with this file descriptor will return at this point, with this descriptor marked as the reason (or one of the reasons) the function has returned. The user does not need to release the obtained file descriptor.

The wake-up mechanism exists to allow for the user process to register for notifications on events of the underlying interfaces, and wait until such occur. This is an alternative to repeated polling for request completion. The goal is to allow for waiting while consuming minimal resources from the system. This is recommended for cases where traffic is infrequent, and latency can be traded for lower resource consumption while waiting for it.

There are two alternative ways to use the wakeup mechanism: the first is the file descriptor obtained per worker (this function) and the second is the ucp_worker_wait function for waiting on the next event internally.

Note

UCP features have to be triggered with UCP_FEATURE_WAKEUP to select proper transport

Parameters

[in]	worker	Worker of notified events.
[out]	fd	File descriptor.

Returns

Error code as defined by ucs_status_t

Examples

ucp_hello_world.c.

ucp_worker_wait()

ucs_status_t ucp_worker_wait

(

ucp_worker_h

worker

)

This routine waits (blocking) until an event has happened, as part of the wake-up mechanism.

This function is guaranteed to return only if new communication events occur on the worker. Therefore one must drain all existing events before waiting on the file descriptor. This can be achieved by calling ucp_worker_progress repeatedly until it returns 0.

There are two alternative ways to use the wakeup mechanism. The first is by polling on a per-worker file descriptor obtained from ucp_worker_get_efd. The second is by using this function to perform an internal wait for the next event associated with the specified worker.

Note

During the blocking call the wake-up mechanism relies on other means of notification and may not progress some of the requests as it would when calling ucp_worker_progress (which is not invoked in that duration).

UCP features have to be triggered with UCP_FEATURE_WAKEUP to select proper transport

Parameters

[in]

worker

Worker to wait for events on.

Returns

Error code as defined by ucs_status_t

Examples

ucp_hello_world.c.

ucp_worker_wait_mem()

void ucp_worker_wait_mem	(	ucp_worker_h	worker,
		void *	address )

This routine waits for a memory update at the local memory address. This is a blocking routine. The routine returns when the memory address is updated ("write") or an event occurs in the system.

This function is guaranteed to return only if new communication events occur on the worker or address is modified. Therefore one must drain all existing events before waiting on the file descriptor. This can be achieved by calling ucp_worker_progress repeatedly until it returns 0.

Note

This routine can be used by an application that executes busy-waiting loop checking for a memory update. Instead of continuous busy-waiting on an address the application can use ucp_worker_wait_mem, which may suspend execution until the memory is updated. The goal of the routine is to provide an opportunity for energy savings for architectures that support this functionality.

Parameters

[in]	worker	Worker to wait for updates on.
[in]	address	Local memory address

ucp_worker_arm()

ucs_status_t ucp_worker_arm

(

ucp_worker_h

worker

)

This routine needs to be called before waiting on each notification on this worker, so will typically be called once the processing of the previous event is over, as part of the wake-up mechanism.

The worker must be armed before waiting on an event (must be re-armed after it has been signaled for re-use) with ucp_worker_arm. The events triggering a signal of the file descriptor from ucp_worker_get_efd depend on the interfaces used by the worker and defined in the transport layer, and typically represent a request completion or newly available resources. It can also be triggered by calling ucp_worker_signal .

The file descriptor is guaranteed to become signaled only if new communication events occur on the worker. Therefore one must drain all existing events before waiting on the file descriptor. This can be achieved by calling ucp_worker_progress repeatedly until it returns 0.

void application_initialization() {
// should be called once in application init flow and before
// process_communication() is used
    ...
    status = ucp_worker_get_efd(worker, &fd);
    ...
}
 
void process_communication() {
// should be called every time need to wait for some condition such as
// ucp request completion in sleep mode.
 
    for (;;) {
        // check for stop condition as long as progress is made
        if (check_for_events()) {
             break;
        } else if (ucp_worker_progress(worker)) {
             continue;                 // some progress happened but condition not met
        }
 
        // arm the worker and clean-up fd
        status = ucp_worker_arm(worker);
        if (UCS_OK == status) {
            poll(&fds, nfds, timeout);  // wait for events (sleep mode)
        } else if (UCS_ERR_BUSY == status) {
            continue;                   // could not arm, need to progress more
        } else {
            abort();
        }
    }
}

Note

UCP features have to be triggered with UCP_FEATURE_WAKEUP to select proper transport

Parameters

[in]

worker

Worker of notified events.

Returns

UCS_OK The operation completed successfully. File descriptor will be signaled by new events.

UCS_ERR_BUSY There are unprocessed events which prevent the file descriptor from being armed. These events should be removed by calling ucp_worker_progress(). The operation is not completed. File descriptor will not be signaled by new events.

Other different error codes in case of issues.

Examples

ucp_hello_world.c.

ucp_worker_signal()

ucs_status_t ucp_worker_signal

(

ucp_worker_h

worker

)

This routine signals that the event has happened, as part of the wake-up mechanism. This function causes a blocking call to ucp_worker_wait or waiting on a file descriptor from ucp_worker_get_efd to return, even if no event from the underlying interfaces has taken place.

Note

It's safe to use this routine from any thread, even if UCX is compiled without multi-threading support and/or initialized with any value of ucp_params_t::mt_workers_shared and ucp_worker_params_t::thread_mode parameters

Parameters

[in]

worker

Worker to wait for events on.

Returns

Error code as defined by ucs_status_t