Common Components

Classes that are used by multiple felix-star executables are presented below.

Device class

The abstract device class is here presented. The derived classes are described in the complete Doxygen documentation.

class Device

Abstract base class that represents a Felix device. The two implementations are FlxDevice and FileDevice. FlxDevice manages one PCIe endpoint of a physical FELIX card. FileDevice emulates the hardware.

Subclassed by FileDevice, FlxDevice

Public Functions

Device(unsigned dev_no, uint8_t vid, uint16_t did, uint16_t cid)

Constructor of abstract class to be used by implementation. The constructor allocates the object, does not interact with hardware.

Parameters:

dev_no -- Device number, as use dby the flx driver.
vid -- Version of felix resource identifier (fid)
did -- Detector ID of felix resource identifier
cid -- Connect ID of felix resource identifier, maps to the PCIe endpoint

virtual ~Device() = default: Destructor of abstract class.

virtual int open_device(u_int lock_mask) = 0

Opens the device, maps and reads registers, sets resource locks.

Parameters:: lock_mask -- to lock resources at driver level

virtual void close_device() = 0: Releases the device.

virtual bool is_primary() = 0

Returns:: whether if this PCIe endpoint is the primary, where all registers are available.

virtual unsigned int get_card_model() = 0

Returns:: the card type: 709, 710, 711, 712, 182, 155.

virtual unsigned int get_card_endpoints() = 0

Returns:: number of PCIe endpoints for the card

virtual u_int get_regmap_version() = 0

Returns:: the firmware register map version, read from the register REG_MAP_VERSION

inline virtual u_int get_firmware_type()

Returns:: the firmware type as reported in the register FIRMWARE_MODE

virtual u_int get_number_of_channels() = 0

Returns:: the number of links for the PCie endpoint as reported in the register NUM_OF_CHANNELS

virtual bool get_wide_mode() = 0

Returns:: whether GBT wide mode is used

virtual u_int get_block_size() = 0

Returns:: the ToHost block size as read from the BLOCKSIZE register

virtual u_int get_trailer_size() = 0

Returns:: the ToHost trailer/header size. Derived from CHUNK_TRAILER_32B.

virtual flx_fromhost_format get_fromhost_data_format() = 0

Returns:: the FromHost data format, as in FROMHOST_DATA_FORMAT.

virtual flx_tohost_format get_tohost_data_format() = 0

Returns:: the ToHost data format, as in TOHOST_DATA_FORMAT.

virtual int get_fromhost_dmaid() = 0

Returns:: the identifier of the FromHoist DMA buffer, to be used by felix-toflx.

virtual int get_trickle_dmaid() = 0

Returns:: the identifier of the Trickle configuration DMA buffer.

virtual int dma_max_tlp_bytes() = 0

Returns:: the maximum size supported for a TLP block

virtual void set_register(const char *key, uint64_t value) = 0

Set a new value for the given register

Parameters:

key -- register name
value -- register value

virtual uint64_t get_register(const char *key) = 0

Get the current value of the fiven register

Parameters:: key -- register name

virtual bool check_register(const char *key) = 0

Parameters:: key -- register name
Returns:: if the given register exists

virtual void cancel_irq(int i) = 0

Cancel wait requests for the given MSI-X interrupt

Parameters:: i -- Interrupt number

virtual void irq_wait(int i) = 0

Wait for the given interrupt. This function blocks.

Parameters:: i -- Interrupt number

virtual void irq_enable(int i) = 0

Arm the given interrupt.

Parameters:: i -- interrupt number

virtual void irq_disable(int i) = 0

Disarm the given interrupt.

Parameters:: i -- interrupt number

virtual std::vector<Elink> read_enabled_elinks(int dmaid) = 0

Return the list of enabled e-links for the given DMA id.

Parameters:: dmaid -- dma buffer identifier

std::vector<Elink> get_enabled_elinks_of_type(int dmaid, elink_type_t t)

Return the list of e-links enabled of a given type.

Parameters:

dmaid -- dma buffer identifier
t -- e-link type e.g. DAQ, DCS, TTC2H...

inline unsigned get_device_number()

Returns:: the device number.

unsigned int make_dma_lock_mask(const std::vector<int> &dmaids)

Parameters:: dmaids -- list of DMA buffers in use
Returns:: the lock mask for DMA resources

u_int get_broadcast_elinks(uint64_t broadcast_elinks[], uint32_t num_of_channels)

Parameters:

broadcast_elinks -- list of DMA buffers in use
num_of_channels -- list of DMA buffers in use

Returns:

the number of broadcast e-links

bool has_wrong_rm(): Compare the software register map version with hardware one

bool is_full_mode(): Firmware does not implement ToHost (lp)GBT e-links

bool is_lpgbt_mode(): Firmware implements ToHost (lp)GBT e-links

virtual uint32_t get_channel(uint32_t elink) = 0

Firmware implements ToHost (lp)GBT e-links

Parameters:: elink -- elink number

virtual uint32_t get_egroup(uint32_t elink, bool is_to_flx) = 0

Firmware implements ToHost (lp)GBT e-links

Parameters:

is_to_flx -- true if the DMA is a FromHost one
elink -- elink number

virtual uint32_t get_epath(uint32_t elink, bool is_to_flx) = 0: Firmware implements ToHost (lp)GBT e-links

u_int get_ctrl_elinks_per_channel(): Firmware implements ToHost (lp)GBT e-links

virtual bool dma_enabled(int dmaid) = 0

Parameters:: dmaid -- true if the DMA is a FromHost one
Returns:: true is the DMA buffer is enabled

virtual void dma_to_host(DmaBuffer *buf, int dmaid) = 0

set up ToHost (circular) DMA buffer

Parameters:: dmaid -- DMA buffer number

virtual void dma_from_host(DmaBuffer *buf, int dmaid) = 0

set up FromHost (circular) DMA buffer

Parameters:: dmaid -- DMA buffer number

virtual void dma_from_host_trickle(DmaBuffer *buf, int dmaid, size_t config_size) = 0

Set up FromHost (circular) DMA buffer for trickle mode.

Parameters:

buf -- DMA buffer pointer.
dmaid -- DMA buffer identifier.
config_size -- Size of the trickle configuration message.

virtual void dma_set_oneshot(DmaBuffer *buf, int dmaid, size_t config_size) = 0

set up FromHost (non-circular) DMA buffer

Parameters:: dmaid -- DMA buffer number

virtual uint64_t dma_get_fw_ptr(int dmaid) = 0

Parameters:: dmaid -- DMA buffer number
Returns:: pointer managed by firmware (wr for ToHost, rd for FromHost)

virtual uint64_t dma_get_sw_ptr(int dmaid) = 0

Parameters:: dmaid -- DMA buffer number
Returns:: pointer managed by software (rd for ToHost, wr for FromHost)

virtual void dma_set_sw_ptr(int dmaid, uint64_t p_absolute) = 0

set new value for pointer managed by software (rd for ToHost, wr for FromHost)

Parameters:

dmaid -- DMA buffer number
p_absolute -- birtual memory address

virtual bool dma_cmp_even_bits(int dmaid) = 0

Parameters:: dmaid -- DMA buffer number
Returns:: whether sw and fw pointers are have wrapped around the circular buffer the the same number of times.

virtual void dma_stop(unsigned int dmaid) = 0

stop DMA buffer data transfer operations

Parameters:: dmaid -- DMA buffer number

virtual monitoring_data_t hw_get_monitoring_data(unsigned int mon_mask = 0xFFF) = 0

Returns:: hardware monitoring data

Public Static Functions

static int64_t broadcast_for(uint64_t elink)

Parameters:: elink -- list of DMA buffers in use
Returns:: (?)

static int broadcast_type(uint64_t elink)

Parameters:: elink -- broadcast e-link
Returns:: the level of broadcast: link, channel, e-liink

static const std::string broadcast_type_str(uint64_t elink)

Parameters:: elink -- broadcast e-link
Returns:: the level of broadcast: link, channel, e-liink

static u_int broadcast_elinks(uint64_t broadcast_elinks[], const uint64_t broadcast_enable_registers[], uint32_t num_of_channels)

read broadcast_elinks and broadcast_enable_registers

Parameters:

broadcast_elinks -- array of broadcast enabled e-links per link
broadcast_enable_registers -- register values for each link
num_of_channels -- number of device links

Returns:

the number of broadcast links

static const std::string get_elink_type_str(elink_type_t encoding)

Return the e-link type given the encoding used

Parameters:: encoding -- e-link encoding

DMA buffers

class DmaBuffer

Base class for a FELIX DMA buffer. The buffer is defined by a physical address (paddr - used by the FELIX card), a size (size) and end-address (pend). The physical address is mapped to a virtual address (vaddr) used by software in user space. Software and firmware maintain each a read/write address depending on the data transfer direction. The address managed by software is pc_ptr. The firmware pointer is read from the card register or is emulated in software by emu_fw_ptr.

Subclassed by CmemBuffer, VmemBuffer

Public Functions

inline explicit DmaBuffer(size_t c)

Constructor for a DMA buffer.

All address are initialised to zero.

Parameters:: size -- buffer size.

Network wrappers

Interface classes are defined to describe the required functionality of a network publisher and receiver while abstracting the implementation.

class Publisher

Interface class to any publisher implemented by the network backend.

Subclassed by Netio3BufferedPublisher, Netio3ZerocopyPublisher

Public Functions

virtual Result publish(felix_id_t fid, std::span<iovec> iovs, size_t bytes, uint32_t block_addr, std::uint8_t status, std::optional<size_t> max_msg_size = std::nullopt) = 0

publish chunk.

Parameters:

fid -- e-link universal identifier.
iovs -- vector containing iovs to send.
bytes -- total message size in bytes.
block_addr -- address of block where the chunk starts (for zero-copy mode).
status -- felix-star status byte.
max_msg_size -- maximum message size for this call. If not set, the message is bound only by the maximum message size supported by the network, which also caps any larger limit. Larger messages are truncated.

Returns:

publisher-specific return code.

virtual Result publish(felix_id_t fid, uint8_t *data, size_t len) = 0

publish chunk/message.

Parameters:

fid -- e-link universal identifier.
data -- pointer to the start of chunk/message to publish.
len -- size of message/chunk to publish.

Returns:

publisher-specific return code.

virtual Result flush(felix_id_t fid) = 0

send right away (flush) the current buffer.

Parameters:: fid -- e-link universal identifier. All buffers containing data from this e-link are flushed.
Returns:: publisher-specific return code.

virtual void set_periodic_callback(uint32_t period_us, Callback callback) = 0: configure periodic callback e.g. the function used for polling input data.

virtual void set_asynch_callback(Callback callback) = 0: configure a callback with an asynchronous signal associated.

virtual void fire_asynch_callback() = 0: fire the signal associated to the asynchronous callback.

virtual const CompletionTable *get_completion_table() = 0

zero-copy specific function to determine the DMA read pointer on the basis of outstanding network transfers.

Returns:: pointer to completion table.

virtual uint32_t get_resource_counter() = 0

Returns:: the number of available network buffers or similar resources.

virtual uint32_t get_subscription_number() = 0

Returns:: the number of subscriptions.

virtual uint64_t get_resource_available_calls() = 0

Returns:: the number of calls occurring when network resources become availale after exhaustion.

virtual uint16_t get_port() const = 0

Get the port number assigned by the system.

Returns:: the port number.

class Receiver

Interface class to any receiver implemented by the network backend.

Subclassed by Netio3Receiver

Public Functions

virtual void set_conn_open_callback(OnConnOpen callback) = 0

application callback for new connections.

Parameters:: callback -- the std::function<void (std::string &)> to invoked when a new connection is established.

virtual void set_conn_close_callback(OnConnClose callback) = 0

application callback for closed connections.

Parameters:: callback -- the std::function<void (std::string &)> to be invoked when a new connection is closed.

virtual void set_on_msg_callback(OnMsg callback) = 0

application callback for received messages.

Parameters:: callback -- the std::function<void(uint8_t*, size_t)> to invoke when a message is received.

virtual int get_number_of_connections() = 0

Monitor the number of open connetions.

Returns:: the number of open connections.

virtual uint16_t get_port() const = 0

Get the port number assigned by the system.

Returns:: the port number.

Felix-bus wrapper

Felix-bus is wrapped by a class that thus defines the interface with the rest of the code. In this way, changes to the felix-bus package would be affect only this part of the code.

class Bus

Inteface class for felix-bus (medium for e-link advertisement)

Public Functions

explicit Bus(const std::string &bus_dir, const std::string &bus_group, int dmaid, bool verbose)

Felix-bus constructor. The bus filename is generated automatically using the DMA buffer identifier.

Parameters:

bus_dir -- bus directory.
bus_group -- bus group name.
dmaid -- DMA buffer number.
verbose -- option for the felix-bus.

explicit Bus(const std::string &bus_dir, const std::string &bus_group, const std::string &filename, bool verbose)

Felix-bus constructor with ad-hoc filename.

Parameters:

bus_dir -- bus directory.
bus_group -- bus group name.
filename -- bus file name, to set a custom file name.
verbose -- option for the felix-bus.

bool publish(std::span<const felixbus::FelixBusElinkInfo> data)

Publish on the current bus a list of e-links

Parameters:: data -- vector of e-links with information to publish.

Monitoring data

Monitoring data produced by applications is JSON encoded and printed to a UNIX FIFO. The content of the JSON string is application-specific but I/O aspects are common.

class Monitor

Generic class to write JSON strings containing monitoring information in UNIX FIFOs.

Subclassed by FromHostMonitor, ToHostMonitor

Public Functions

inline explicit Monitor(std::unique_ptr<Writer> writer)

Constructor to initialize the Monitor with a Writer.

Parameters:: writer -- Unique pointer to a Writer object.

virtual ~Monitor() = default: Virtual destructor.

inline std::string get_serialized_message()

Get the serialized JSON message as a string.

Returns:: Serialized JSON message.

inline void write_message(): Write the JSON message using a Writer.

inline void create_new_message(const std::string &hostname)

Create a new JSON message, initializing it with the given timestamp and hostname.

Parameters:: hostname -- The hostname for the new message.

I/O with permanent storage

Emulation of FELIX hardware can require reading or writing in files or UNIX FIFOs. A class has been implement to simplify the task.

class DiskIO

Class used to read from or write into files or Unix fifos. Used in tests when hardware is emulated in software.

Public Functions

DiskIO(const std::string &filename, size_t block_size, std::string_view op, bool is_fifo)

DiskIO constructor.

Parameters:

filename -- file or fifo to open.
block_size -- felix block size.
op -- read or write.
is_fifo -- true if fifo, file otherwise.

size_t block_read(void *to_read, size_t max_blocks)

read blocks from file/fifo.

Parameters:

to_read -- read from stream to this destination.
max_blocks -- to read.

Returns:

number of blocks read.

size_t block_write(const void *to_write, size_t max_blocks)

write blocks into file/fifo.

Parameters:

to_write -- write this data into the output stream.
max_blocks -- to write.

Returns:

number of blocks written.

size_t byte_read(void *to_read, size_t bytes)

read bytes from file/fifo.

Parameters:

to_read -- read from file/fifo to this destination.
bytes -- maximum number of bytes to read.

Returns:

number of bytes read.

size_t byte_write(void *to_write, size_t bytes)

write bytes to file/fifo.

Parameters:

to_write -- write this data into the fifo/file.
bytes -- maximum number of bytes to write.

Returns:

number of bytes written.

bool is_eof()

Returns:: if the end-of-file has been reached.

int reset_position()

Returns:: go back to the start of file.