Q.ANT native computing toolkit

All you need for photonic computing.

Installation

Install this package from the provided wheel via pip. To use the Q.ANT native computing toolkit, you need to have the Q.ANT native computing driver installed on your system. You can install the debian package of the installer from the provided .deb file via apt.

API documentation

The Python API is grouped into five submodules:

qant_native_computing_toolkit.generic
qant_native_computing_toolkit.native
qant_native_computing_toolkit.ai
qant_native_computing_toolkit.info
qant_native_computing_toolkit.utils

Generic functionality

qant_native_computing_toolkit.generic.get_available_npus() → dict

Lists all available NPUs on the system.

Args:

Returns:: The mapping from device_id to serial number
Return type:: npus (dict[int, str])

qant_native_computing_toolkit.generic.init_npu(id: int) → None

Initialises the Native Processing Unit. Optional, not necessary for computation.

Parameters:: id – The identifier of the NPU that is to be initialised.

qant_native_computing_toolkit.generic.release_npu(id: int) → None

Releases an NPU and frees associated memory. Only relevant if multiple users want to access the same device.

Parameters:: id – The identifier of the NPU that is to be released.

Native math functionality

qant_native_computing_toolkit.native.calc_scaled_periodic_nl_fprop(features: ndarray, weights: ndarray, device_id: int = 0) → ndarray

Calculates a scaled periodic nonlinearity pairwise for all elements of features \(u\) and weights \(v\). For each input pair, the output is

\[w(u, v) = \operatorname{tcos}(u) \cdot v\]

where \(\operatorname{tcos}\) is a \(2\pi\)-periodic function with values between \(-1\) and \(1\), similar to a cosine.

Parameters:

features (np.ndarray) – A 1D array of input features. Must be dtype=bfloat16.
weights (np.ndarray) – A 1D array of weights. Must be dtype=bfloat16.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

A 1D array containing the result, same shape as input features.

Return type:

np.ndarray

qant_native_computing_toolkit.native.linear_fprop(features: ndarray, weights: ndarray, device_id: int = 0) → ndarray

Performs a linear forward propagation between input features and a weight matrix. Operation is performed along the last dimension of features. Additional dimensions are treated as batch dimensions.

The operation computes output = features @ weights.T, i.e.:

\[\text{output}[i, j] = \sum_k \text{features}[i, k] \cdot \text{weights}[j, k]\]

Note that this is not a plain features @ weights — the weight matrix is implicitly transposed, matching the convention where weights has shape (n_channels_out, n_channels_in).

Parameters:

features (np.ndarray) – An ND array of input features with shape (<batch dimensions>, n_channels_in). Must be dtype=bfloat16.
weights (np.ndarray) – A 2D array representing the weights with shape (n_channels_out, n_channels_in). Must be dtype=bfloat16.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array containing the result of the forward propagation with shape (<batch dimensions>, n_channels_out).

Return type:

np.ndarray

qant_native_computing_toolkit.native.mul_elementwise(us: ndarray, vs: ndarray, device_id: int = 0) → ndarray

Multiply two arrays element-wise.

Parameters:

us (np.ndarray) – First input array, must be of type bfloat16 and have same shape as vs.
vs (np.ndarray) – Second input array, must be of type bfloat16 and have same shape as us.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

A new array of the same shape as us and vs, containing the element-wise product of the two input arrays, with type bfloat16.

Return type:

np.ndarray

qant_native_computing_toolkit.native.mul_f32(us: ndarray, vs: ndarray, device_id: int = 0) → ndarray

Multiply two one-dimensional arrays element-wise.

Parameters:

us (np.ndarray) – First input array, must be of type np.float32.
vs (np.ndarray) – Second input array, must be of type np.float32.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

A new array of the same shape as us and vs, containing the element-wise product of the two input arrays, with type np.float32.

Return type:

np.ndarray

qant_native_computing_toolkit.native.mul_i16(us: ndarray, vs: ndarray, device_id: int = 0) → ndarray

Multiply two arrays element-wise.

Parameters:

us (np.ndarray) – First input array, must be of type np.int16 and have same shape as vs.
vs (np.ndarray) – Second input array, must be of type np.int16 and have same shape as us.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

A new array of the same shape as us and vs, containing the element-wise product of the two input arrays, with type np.int16.

Return type:

np.ndarray

AI functionality

qant_native_computing_toolkit.ai.adaptive_avgpool2d_fprop(features: ndarray, output_size: int | tuple[int, int], device_id: int = 0) → ndarray

Performs a forward pass through an adaptive avgpooling2d layer. Only works for symmetric input and output sizes. Input size has to be integer multiple of output size.

Parameters:

features (np.ndarray) – A 3D array of input features with shape (n_channels, height, width) or a 4D array of input features with the shape (batches, n_channels, height, width). Must be dtype=bfloat16.
output_size (int | (int, int)) – Pooling window dimensions. If int, same size is used for both dimensions.
device_id (int, optional) – The identifier of the device on which the operation is performed.

Returns:

An ND array containing the result of the adaptive avgpool operation with shape ([batches], n_channels, output_height, output_width).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.adaptive_maxpool2d_fprop(features: ndarray, output_size: int | tuple[int, int], device_id: int = 0) → ndarray

Performs a forward pass through an adaptive maxpooling2d layer. Only works for symmetric input and output sizes. Input size has to be integer multiple of output size.

Parameters:

features (np.ndarray) – A 3D array of input features with shape (n_channels, height, width) or a 4D array of input features with the shape (batches, n_channels, height, width). Must be dtype=bfloat16.
output_size (int | (int, int)) – Pooling window dimensions. If int, same size is used for both dimensions.
device_id (int, optional) – The identifier of the device on which the operation is performed.

Returns:

An ND array containing the result of the adaptive maxpool operation with shape ([batches], n_channels, output_height, output_width).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.add_bias_fprop(features: ndarray, bias: ndarray, device_id: int = 0) → ndarray

Performs a forward pass through a bias adder. Aka element-wise addition. If 3D features are used, this function assumes the layout (n_channels, height, width)

Parameters:

features (np.ndarray) – An array of 1 to 4 dimensions of input features with shape ([batches], n_channels, [height, width]). Must be dtype=bfloat16.
bias (np.ndarray) – A 1D array of bias values with shape (n_channels). Must be dtype=bfloat16.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

A 1-4D array of input features with shape ([batches], n_channels, [height, width]) containing the result of the addition.

Return type:

np.ndarray

qant_native_computing_toolkit.ai.avgpool2d_fprop(features: ndarray, kernel_size: int | tuple[int, int], stride: int, padding: int, count_include_pad: bool = True, device_id: int = 0) → ndarray

Apply 2D avg pooling operation to input features using specified parameters.

Parameters:

features (np.ndarray) – A 3D array of input features with shape (n_channels, height, width) or a 4D array of input features with the shape (batches, n_channels, height, width). Must be dtype=bfloat16.
kernel_size (int | (int, int)) – Pooling window dimensions. If int, same size is used for both dimensions.
stride (int) – Step size for sliding pooling window (must be non-negative).
padding (int) – Zero-padding size to apply to input boundaries (must be non-negative).
count_include_pad (bool, optional) – When True, will include the zero-padding in the averaging calculation.
device_id (int, optional) – The identifier of the device on which the operation is performed.

Returns:

An ND array containing the result of the avgpool operation with shape ([batches], n_channels, output_height, output_width).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.batchnorm2d_fprop(features: ndarray, means: ndarray, variances: ndarray, weights: ndarray, bias: ndarray, eps: int, device_id: int = 0) → ndarray

Performs a forward pass through a batchnorm2d layer

Parameters:

features (np.ndarray) – A 3D array of input features with shape (n_channels, height, width) or a 4D array of input features with the shape (batches, n_channels, height, width). Must be dtype=bfloat16.
means (np.ndarray) – A 1D array containing the mean values for each input feature.
variances (np.ndarray) – A 1D array containing the variance values for each input feature.
weights (np.ndarray) – A 1D array containing the scale parameters for each input feature.
bias (np.ndarray) – A 1D array containing the bias parameters for each input feature.
eps (float) – A small value added to the variance for numerical stability.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array containing the result of the batchnorm operation with shape ([batches], n_channels, height, width).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.calc_kan_layer_fprop(features: ndarray, phis: ndarray, ampls: ndarray, ks: ndarray, device_id: int = 0) → ndarray

Calculates a Q.ANT version of a KAN layer (https://arxiv.org/abs/2404.19756) based on calc_scaled_periodic_nl_fprop. The mathematical function is

\[y_j = \sum_{i, l} \operatorname{tcos}(\mathrm{ks}_l \cdot x_i + \phi_{jil}) \cdot \mathrm{ampls}_{jil},\]

where \(x_i\) is the input (vector), \(\mathrm{ks}\) the frequency components, \(\phi\) the phase offsets (tensor), \(\mathrm{ampls}\) the amplitude (tensor), and \(y_j\) the output (vector). \(\operatorname{tcos}\) denotes the cosine-related shape of the periodic optical nonlinearity.

Parameters:

features (np.ndarray) – An ND input array with shape (<batch dimensions>, n_channels_in), dtype=bfloat16. The operation is performed along the last axis, additional dimensions are treated as batch dimensions.
phis (np.ndarray) – A 3D array of phase offsets, shape (n_channels_out, n_channels_in, len(ks)), dtype=bfloat16.
ampls (np.ndarray) – A 3D array of amplitudes, same shape as phis, dtype=bfloat16.
ks (np.ndarray) – A 1D array of frequency components, dtype=bfloat16.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

The result of the KAN layer with shape (<batch dimensions>, n_channels_out), dtype=bfloat16.

Return type:

np.ndarray

qant_native_computing_toolkit.ai.conv_fprop(features: ndarray, kernels: ndarray, padding: int, stride: int, dilation: int, device_id: int = 0) → ndarray

Performs a forward pass through a convolution layer.

Parameters:

features (np.ndarray) – A 3D array of input features with shape (n_channels_in, height, width) or a 4D array of input features with the shape (batches, n_channels_in, height, width). Must be dtype=bfloat16.
kernels (np.ndarray) – A 4D array representing the filter with shape (n_channels_out, n_channels_in, height, width). Must be dtype=bfloat16.
padding (int) – Amount of padding added to the input features.
stride (int) – Step size for moving the filter window over the input features.
dilation (int) – Dilation (“zoom out”) of the filter window.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array containing the result of the convolution with shape ([batches], n_channels_out, height, width).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.conv_transpose_fprop(features: ndarray, kernels: ndarray, padding: int, stride: int, dilation: int, output_padding: int, device_id: int = 0) → ndarray

Performs a forward pass through a transposed convolution layer.

Parameters:

features (np.ndarray) – A 3D array of input features with shape (n_channels_in, height, width) or a 4D array of input features with the shape (batches, n_channels_in, height, width). Must be dtype=bfloat16.
kernels (np.ndarray) – A 4D array representing the filter with shape (n_channels_in, n_channels_out, height, width). Must be dtype=bfloat16.
padding (int) – Amount of padding added to the input features.
stride (int) – Step size for moving the filter window over the input features.
dilation (int) – Dilation (“zoom out”) of the filter window.
output_padding (int) – Padding for the returned features.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array containing the result of the transpose convolution with shape ([batches], n_channels_out, height, width).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.linear_fprop(features: ndarray, weights: ndarray, device_id: int = 0) → ndarray

Performs a linear forward propagation between input features and a weight matrix. Operation is performed along the last dimension of features. Additional dimensions are treated as batch dimensions.

The operation computes output = features @ weights.T, i.e.:

\[\text{output}[i, j] = \sum_k \text{features}[i, k] \cdot \text{weights}[j, k]\]

Note that this is not a plain features @ weights — the weight matrix is implicitly transposed, matching the convention where weights has shape (n_channels_out, n_channels_in).

Parameters:

features (np.ndarray) – An ND array of input features with shape (<batch dimensions>, n_channels_in). Must be dtype=bfloat16.
weights (np.ndarray) – A 2D array representing the weights with shape (n_channels_out, n_channels_in). Must be dtype=bfloat16.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array containing the result of the forward propagation with shape (<batch dimensions>, n_channels_out).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.maxpool2d_fprop(features: ndarray, kernel_size: int | tuple[int, int], stride: int, padding: int, device_id: int = 0) → ndarray

Performs a forward pass through a maxpooling2d layer

Parameters:

features (np.ndarray) – A 3D array of input features with shape (n_channels, height, width) or a 4D array of input features with the shape (batches, n_channels, height, width). Must be dtype=bfloat16.
kernel_size (int | (int, int)) – Pooling window dimensions. If int, same size is used for both dimensions.
stride (int) – Step size for sliding pooling window (must be non-negative).
padding (int) – Zero-padding size to apply to input boundaries (must be non-negative).
device_id (int, optional) – The identifier of the device on which the operation is performed.

Returns:

An ND array containing the result of the maxpool operation with shape ([batches], n_channels, output_height, output_width).

Return type:

np.ndarray

qant_native_computing_toolkit.ai.relu_fprop(features: ndarray, device_id: int = 0) → ndarray

Performs a forward pass through a ReLU layer.

Parameters:

features (np.ndarray) – An ND array of input features. Must be dtype=bfloat16.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array containing the result of the ReLU operation, same shape as input features.

Return type:

np.ndarray

qant_native_computing_toolkit.ai.sigmoid_fprop(features: ndarray, device_id: int = 0) → ndarray

Performs a forward pass through a Sigmoid layer.

Parameters:

features (np.ndarray) – An ND array of input features. Must be dtype=bfloat16.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array containing the result of the Sigmoid operation with the same shape as the input.

Return type:

np.ndarray

qant_native_computing_toolkit.ai.softmax_fprop(features: ndarray, device_id: int = 0) → ndarray

Performs a forward pass through a Softmax layer.

Parameters:

features (np.ndarray) – An ND array of input features. Must be dtype=bfloat16. If multidimensional, the first dimension must be 1.
device_id (int, optional) – The identifier of the NPU on which the operation is performed.

Returns:

An ND array, containing the result of the Softmax operation with the same shape as the input.

Return type:

np.ndarray

Information and logging

qant_native_computing_toolkit.info.get_driver_info(device_id: int = 0) → str

Gets detailed information about the Q.ANT native computing driver and its direct dependencies.

Parameters:: device_id (int, optional) – The identifier of the device on which the operation is performed.
Returns:: The information as a string

qant_native_computing_toolkit.info.get_perf_counter(device_id: int) → dict

Gets the performance counters.

Parameters:: device_id (int) – The identifier of the device on which the operation is performed.
Returns:: ‘timebased_counter’: Seconds since the last reset call. ‘stalling_counter’: Seconds since the last reset call spent in idle (stalling) mode.
Return type:: The information as a dictionary

qant_native_computing_toolkit.info.get_sensor_info(device_id: int) → dict

Get detailed information about the NPU sensors, e.g. temperature.

Parameters:: device_id (int) – The identifier of the device on which the operation is performed.
Returns:: The information as a dictionary.

qant_native_computing_toolkit.info.get_version_info(device_id: int) → dict

Gets information about the firmware versions.

Parameters:: device_id (int) – The identifier of the device on which the operation is performed.
Returns:: The version information as a dictionary.

qant_native_computing_toolkit.info.reset_perf_counter(device_id: int)

Reset the performance counter.

Parameters:: device_id (int) – The identifier of the device on which the operation is performed.

Returns:

qant_native_computing_toolkit.info.setup_logging(folder_name: str | Path, loglevel: int)

Enable logging. Log messages are written to stdout and to a file

Parameters:

folder_name (str or pathlib.Path) – The name of the folder in which the logfiles are stored. Individual log files are created in this folder and identified by timestamp.
loglevel (int) – The log level from 1 = debug to 4 = error.

Returns:

Helper functions

qant_native_computing_toolkit.utils.align_ndarray_page_boundary(data: ndarray)

Return a copy of a NumPy array whose underlying data buffer is aligned to a page-sized memory boundary.

Parameters:: data (np.ndarray) – NumPy array which has to be aligned.
Returns:: A new NumPy array with the same shape and dtype as data aligned to PAGE_ALIGNMENT bytes.

Memory Layout

The NPU requires all input buffers to be aligned to a page boundary of 4096 bytes. If the provided data is not page-aligned, the runtime will perform an additional copy, which can negatively impact performance. To avoid this unnecessary copy, ensure that your data is already aligned before passing it to the NPU.

You can use the helper function qant_native_computing_toolkit.utils.align_ndarray_page_boundary() to create a page-aligned NumPy array.