felix-toflx

Felix-toflx is the application that receives messages from the network, encodes them into FromHost messages and transfers them to the corresponding FromHost DMA buffer. Felix-toflx can serve multiple devices (each device has a single FromHost DMA buffer).

Top-level Class

The top-level class is FromHost, used to build two executables: felix-toflx that works with a FELIX card and felix-toflx2file that emulates the hardware. FromHost is templated to support both use cases. The template parameters are three: the configuration parameters (emulating hardware requires passing in more parameters), the type of FromHost DMA buffer (that interfaces with the FELIX driver or not) and the type of FELIX device (real or emulated). FromHost contains pointers to devices, DMA buffers and writers.

template<class C, class D, class B>
class FromHost

Class at the core of felix-toflx and felix-file2flx. Templeted over Config, Device and ToHostBuffer classes.

Public Functions

void start()

allocate all resources and start thread(s).

void stop()

stop transfers, join all created threads.

void print_monitoring()

print monitoring information in FIFO.

Writers

FromHostWriter serves a set of FromHost links and owns a network Receiver. The Receiver posts on the Bus the e-links and the corresponding network configuration. The Receiver also runs the event loop for which FromHostWriter functions are registered as callbacks. Network messages are parsed and written to the FRomHost DMA buffer using the FromHost buffer interface.

Note

Felix-toflx uses the send/receive network pattern, not publish/subscribe.

Warning

Network buffers used in the send/receive patter by netio-next do not include any e-link identifier. For this reason each message is preceded by a ToFlxHeader. Netio3 buffer structures will include the e-link identifier and such header will disappear.

template<class B>
class FromHostWriter

FromHostWriter owns a network Receiver that runs the (event loop) thread. Messages are received from the network, encoded, and written into the FromHost buffer. The FromHost can be shared by more than one FromHostWriter. FromHostWriter is templated to support FlxFromHostBuffer and FileFromHostBuffer.

Public Functions

FromHostWriter(std::shared_ptr<B> buffer, const std::vector<Elink> &elinks, std::unique_ptr<Receiver> receiver, bool trickle = false)

FromHostWriter contructor.

Parameters:

• buffer -- shared pointer to Fromhost DMA buffer.

• elinks -- vector of enabled e-links.

• receiver -- pointer to network receiver.

FromHostWriter(std::shared_ptr<B> buffer, elink_type_t elinks_type, std::unique_ptr<Receiver> receiver)

encode and write an incoming network message.

Parameters:

• buffer -- shared pointer to Fromhost DMA buffer.

• elinks_type -- type of e-links to be enabled.

• receiver -- pointer to network receiver.

inline void connection_established(const std::string &s)

on-connection-established callback.

inline void connection_closed(const std::string &s)

on-connection-closed callback.

void encode_and_write(const std::vector<ToFlxMessage> &messages, bool trickle = false)

encode and write an incoming network message.

Parameters:

messages -- vector of ToFlxMessage structs representing incoming network messages.

std::map<local_elink_t, FromHostElinkStats> get_writer_elink_stats()

Returns:

map of e-link / e-link monitoring information.

inline unsigned int get_number_of_connections()

Returns:

number of open connections.

inline uint32_t get_id() const

Returns:

id of the writer

FromHost Buffer

The FromHost DMA buffer is described by the FromHostBuffer class, specialised to the case in which software writes and firmware reads (as opposed to the ToHost case) in a circular buffer. The circular [1] FromHost DMA buffer is supported; the "one-shot" buffer allocated per transaction is no longer supported.

FromHostBuffer is an abstract class from which two implementations derive: one to interface with a FELIX card and another one in which the behaviour of the card is emulated. In the latter case, a dedicated thread is spawned to read the DMA buffer (optionally writing data to a file or UNIX fifo) and advance the read pointer.

A message passed from FromHostWriter for writing is first encoded by the Encoder owned by FromHostBuffer. The write pointer is then advanced according to the size of the encoded message that is typically larger than the original size. The encoding and writing is run by the FromHostWriter thread. A mutex regulates the writer's access to the buffer but it is locked only if multiple writers are used.

[1]

or "continuos" in flxcard jargon.

class FromHostBuffer

Abstract class representing a FromHost DMA buffer. The elinks enabled for the corresponding DMA buffers are read from the device in the constructors.

Subclassed by FileFromHostBuffer, FlxFromHostBuffer

Public Functions

inline FromHostBuffer(std::shared_ptr<Device> d)

FromHostBuffer contructor.

Parameters:

d -- Device shared pointer.

inline std::shared_ptr<Device> get_device()

Returns:

pointer to device in use.

inline size_t get_size() const

Returns:

size of DMA buffer.

inline size_t get_trickle_config_size()

Returns:

size of the trickle configuration message.

inline void set_trickle_config_size(size_t size)

Parameters:

size -- : size of trickle configuration message

inline void set_encoder_data_format(int format)

Parameters:

format -- : FromHost data format supported by firmware.

inline int get_dmaid() const

Returns:

identifier of DMA buffer.

inline void set_dmaid(int dma_id)

Parameters:

dma_id -- identifier of DMA buffer.

Returns:

null.

inline void stop()

Returns:

stop DMA transfer operations.

inline std::vector<Elink> get_elinks()

Parameters:

m_dmaid -- DMA identifier

Returns:

vector of enabled e-links.

inline std::vector<Elink> get_elinks_of_type(elink_type_t t)

Parameters:

m_dmaid -- DMA identifier

Returns:

vector of enabled e-links of a given type (DAQ, DCS...)

inline void increment_writer_counter()

increment the counter of writers concurrently acessing the DMA buffer

inline bool has_multiple_writers() const

Returns:

whether the FromHost DMA buffer is in use by more than a writer.

inline size_t compute_msg_dma_occupancy(size_t size)

Parameters:

size -- of message to be encoded and written.

Returns:

size of the encoded messagesss provided by encoder.

void encode_and_write(uint64_t elink, const uint8_t *source, size_t size, bool trickle = false)

encode and write message in the FromHost buffer.

Parameters:

• elink -- of message to be encoded and written.

• source -- of message to be encoded and written.

• size -- of message to be encoded and written.

inline uint32_t dma_get_free_MB()

Returns:

free space in the FromHost buffer in MB.

FromHostDmaStats get_monitoring_data()

Returns:

monitoring information with rates.

virtual size_t dma_free_bytes() = 0

Returns:

free space in the FromHost buffer in bytes.

virtual void allocate_buffer(size_t size, const std::string &name, bool vmem, bool free_previous_cmem = true) = 0

Parameters:

• size -- of the FromHost DMA buffer.

• name -- of the buffer passed to the CMEM driver.

• vmem -- allocate the buffer without using the CMEM driver.

• free_previous_cmem -- claim an unlocked/orphaned CMEM buffer with the same name.

Encoder

Messages coming from the network must be encoded according to the "Central Rounter FromHost data format" described in the firmware specification document. In short, messages have to be split in chunks of fixed size (the size depends on the FELIX card PCIe generation) preceded by an header that specified the e-link identifier and message size.

class Encoder

Encodes network messages into FromHost blocks.

Public Functions

Encoder() = default

Encoder default contructor.

void set_data_format(int format)

Sets the data format of the encoder.

Parameters:

format -- FromHost data format supported by firmware.

void set_destination_parameters(uint8_t *dest_start, uint32_t dest_size)

set address and size of buffer where encoded messages are written.

Parameters:

• dest_start -- start address of output buffer.

• dest_size -- size of output buffer.

size_t compute_max_msg_occupancy(size_t size) const

Parameters:

size -- input message size.

Returns:

size of encoded message.

int compute_full_blocks(size_t size) const

Parameters:

size -- input message size.

Returns:

number of full FromHost blocks.

size_t compute_final_bytes(size_t size) const

Parameters:

size -- input message size.

Returns:

number of bytes in the last block of the encoded message.

inline size_t get_block_size() const

Returns:

FromHost block size.

uint64_t encode_and_write_msg(uint64_t elink, const uint8_t *source, size_t size, uint32_t dest_offset)

encodes the input and write it in the output buffer.

Parameters:

• elink -- of the incoming message.

• source -- address of the incoming message.

• size -- of the incoming message.

• dest_offset -- current write pointer offset in the output buffer.

Returns:

new write pointer as byte index.

Monitoring

Monitoring information consists of a JSON string containing statistics regarding the encoder, writers, and DMA buffers / devices. The JSON string can be written to a UNIX fifo such that another application can read it and forward it where needed. The FromHostMonitor class provides methods to assemble the JSON message from the inputs. As in the ToHostMonitor case, there is no thread synchronisation in the retrieval of monitoring information. The main application threads copies the counters without resetting them and computes the increment with respect to the previous copy.

class FromHostMonitor : public Monitor

Data structure for the monitoring FromHost transfers of a device. It contains information about the DMA buffer, the associated writers and the e-links served by each writer.