Sending data to felix-toflx

Various send_data functions are provided to send data to felix-toflx. felix-toflx is publishing the necessary parameters to establish network connections on the bus. Before data can be sent, the connection to felix-toflx must be established calling init_send_data. felix-client will re-use existing connections if possible. If a connection is lost (e.g. because felix-toflx went down), the client will automatically attempt to re-establish the connection. There is no function to close connections.

Synchronous vs asynchronous operations

Connections can be established synchronously or asynchronously. The synchronous functions block until the connection is established or a timeout occurs. The asynchronous functions return immediately and informs the user using a callback. Sending data is asynchronous by default.

Establishing a connection

Connections are established using the init_send_data (synchronous) or init_send_data_nb (asynchronous) function.

virtual void FelixClientThreadExtension520::init_send_data(std::uint64_t fid, std::chrono::milliseconds timeout) = 0

Initialize a send connection to FELIX.

This function checks if a connection has to be established and opens a new connection if needed. The function will block until the connection is established or the timeout is reached. If the function returns without throwing an exception, the connection is established.

Throws:
Parameters:

  • fid – The tag to send data to

  • timeout – The timeout for establishing the connection

virtual void FelixClientThreadExtension520::init_send_data_nb(std::uint64_t fid) = 0

Asynchronously initialize a send connection to FELIX.

This function checks if a connection has to be established and opens a new connection if needed. The function will return immediately. The result is communicated through the on_connect callback. If the connection is refused, the on_refused callback is called. If already the connection request fails, an exception is thrown.

Throws:

felix::BusException – if the tag is not found in the bus

Parameters:

fid – The tag to send data to

Both functions can report a felix::BusException if the lookup in the bus failed. The underlying reason is reported by the exception and can be retrieved by calling get_underlying_message.

The synchronous function can also report a felix::SendConnectionTimeoutException if the connection could not be established within the timeout and a felix::SendConnectionRefusedException if the connection was refused by the server.

The asynchronous function should call one of the two callbacks This callback or This callback to inform the user about the result of the connection.

typedef std::function<void(uint64_t fid)> FelixClientThreadInterface::OnConnectCallback
typedef std::function<void(uint64_t fid)> FelixClientThreadExtension520::OnConnectionRefusedCallback

Tip

As for subscriptions, it is recommended to use the synchronous function to establish a connection. The asynchronous should only be used if you have a good reason to do so. However, conpared to subscriptions, handling the asynchronous case is much easier since only a connection needs to be established which should always result in one of the two callbacks being called. The connection is either established or refused. For subscriptions, messages need to be exchanged over the network which has an inherent chance to fail and not cause any callback to be called.

Caution

If you use asynchronous connections, make sure that callbacks are not blocking.

Sending data

Warning

Calling send_data without establishing a connection first is deprecated and will be removed in a future release.

There are two functions to send data to felix-toflx. The first function sends a single message, the second one sends a list of messages.

virtual void FelixClientThreadExtension520::send_data(std::uint64_t fid, std::span<const std::uint8_t> data, bool flush) = 0

Function to send a message to a remote fid.

This function sends a single message to FELIX. If a buffered sender is used (depending on information in the bus), the flush parameter can be used to flush the buffer and send the message immediately.

Note

Can be called without init_send_data but this behavior will change and throw an exception in the future.

Throws:
Parameters:

  • fid – The tag to send data to

  • data – The data to send

  • flush – Flush the buffer

virtual void FelixClientThreadExtension520::send_data(std::uint64_t fid, const std::vector<std::span<const std::uint8_t>> &msgs) = 0

Function to send multiple messages to a remote fid.

This function sends multiple messages to FELIX. If a buffered sender is used (depending on information in the bus) it is the same as calling send_data for each message. For unbuffered sending, messages will still be send together in one transaction (or as slittle as possible to fit into buffers).

Note

Can be called without init_send_data but this behavior will change and throw an exception in the future.

Throws:
Parameters:

  • fid – The tag to send data to

  • data – The data to send

  • flush – Flush the buffer

Both functions take the FID of the stream to send data to and the data to be sent as a span (essentially a pointer with a size). The first function also takes a boolean parameter flush. If flush is set to true, the data is sent immediately. The second function assumes flushes automatically.

Tip

The second one function taking multiple messages ensures that the messages are sent in the same network transaction to avoid issues if message timing is important.

Both functions make a copy of the data to be sent. Therefore, the user can safely modify or delete the data after the function has returned.

Both functions can throw four exceptions indicating failure. If the function returns without exception, the data will be sent:

  1. felix::SendWhileConnectionDownException: the connection to felix-toflx is currently down. It was established at some point, but was lost subsequently. The reason might be that the server is down. felix-client will attempt to reconnect automatically.

  2. felix::ResourceNotAvailableException: no network resources are available to send the data. This can happen data is sent too fast. The user should wait a bit and try again. If the issue persists, there might be an issue with the server.

  3. felix::MessageTooBigException: the message is too big to be sent. The maximum size of a message is defined by the server and specified in the bus. Change the command line parameters of felix-toflx to increase the buffer size.

  4. felix::BusException: the lookup in the bus failed.

Caution

Once the deprecated behavior that send_data can be called without establishing a connection first is removed, the function will throw a felix::SendBeforeOpenException if the connection was not established before calling send_data.

Send methods

The send method is determined by the settings in the bus file which are determined by the felix-toflx command line parameters. In general, RDMA relies on the allocation of network buffers to send data while TCP tries to optimize the memory consumption by allocating memory in demand when using unbuffered sending. Here is a list of the possible send methods:

  • buffered, TCP/RDMA, single message: Copy data into pre-allocated buffers

  • buffered, TCP/RDMA, multiple messages: Copy data into pre-allocated buffers. Flush buffer at the end. Data might be split into multiple network transactions if they do not fit into the current buffer.

  • unbuffered, TCP, single message: Allocate memory on demand. Copy data into the allocated memory. Then send it.

  • unbuffered, TCP, multiple messages: Allocate memory on demand. Copy data into the allocated memory. Flush buffer at the end. Data is guaranteed to be sent in the same network transaction.

  • unbuffered, RDMA, single message: Copy data into pre-allocated buffers. Flush buffer.

  • unbuffered, RDMA, multiple messages: Copy data into pre-allocated buffers. Flush buffer at the end. Data is guaranteed to be sent in the same network transaction.

Closing send connections

There is no function to close the connection to felix-toflx. The connection is closed automatically when the felix-client-thread instance is destroyed.

Deprecated API

Calling send_data without establishing a connection first is deprecated and will be removed in a future release.

All send_data functions taking a pair of const uint8_t* and size_t are deprecated.

All non-blocking send_data functions are deprecated. send_data is now always non-blocking in the sense of the previous non-blocking functions.

Example

 1try {
 2  fct.init_send_data(fid, 1000ms);
 3} catch (const felix::FelixClientException& e) {
 4  std::println("Failed to establish connection: {}", e.what());
 5  return;
 6}
 7
 8const auto data = std::vector<std::uint8_t>{0, 1, 2, 3, 4, 5};
 9
10try {
11  client.send_data(fid, data, true);
12} catch (const felix::FelixClientException& e) {
13  std::println("Failed to send data: {}", e.what());
14  return;
15}

(Incomplete) example using asynchronous connection establishment (exception handling omitted):

 1const auto fid = 0x1000000000080000;
 2
 3std::atomic_bool connected{false};
 4
 5void on_connect_callback(const std::uint64_t fid_connected) {
 6  std::println("Connected to FID: {:#x}", fid_connected);
 7  if (fid_connected == fid) {
 8    connected = true;
 9  }
10}
11
12fct.init_send_data_nb(fid);
13
14connected.wait(false);
15// Questions:
16// - What about timeout?
17// - Cancel wait if the connection was refused?
18// - If using a timeout, what happens if the connection is established after the timeout?
19
20const auto data = std::vector<std::uint8_t>{0, 1, 2, 3, 4, 5};
21client.send_data(fid, data, true);