Version 5.0.0 · Apache-2.0 · C++ and Python

Holoscan Vision

Holoscan Vision packages pipeline-native image operators for Holoscan applications. Eight operators wrap CV-CUDA, and Bayer demosaic is backed by NPP. They work on the tensors already in your graph and submit work on the operator’s CUDA stream.

At a glance

Package	`holoscan-vision`
Version	5.0.0
C++ namespace	`holoscan::vision`
Python namespace	`holoscan.vision`
Holoscan SDK	5.0.0 or later, tested with 5.0.0
Platforms	x86_64, aarch64
Languages	C++, Python
License	Apache-2.0
Maintainer	Holoscan Team, NVIDIA
Debian package	`holoscan-vision`
PyPI package	`holoscan-vision-cu13`

The public operator names match the CV-CUDA operations they wrap, so the CV-CUDA documentation stays useful while you are wiring a Holoscan graph.

Operators

Operator	Registry name	Ports	What you configure
`BayerDemosaicOp`	`holoscan.vision.BayerDemosaicOp`	`input` → `output`	Reconstruct RGB or RGBA from a Bayer mosaic
`ResizeOp`	`holoscan.vision.ResizeOp`	`input` → `output`	Width, height, interpolation
`CustomCropOp`	`holoscan.vision.CustomCropOp`	`input` → `output`	Crop rectangle
`CopyMakeBorderOp`	`holoscan.vision.CopyMakeBorderOp`	`input` → `output`	Border widths, mode, value
`RemapOp`	`holoscan.vision.RemapOp`	`input` + `map` → `output`	Size, interpolation, map mode, map age, border
`NormalizeOp`	`holoscan.vision.NormalizeOp`	`input` → `output`	Base, scale, global scale, shift, epsilon
`ConvertToOp`	`holoscan.vision.ConvertToOp`	`input` → `output`	Output dtype, alpha, beta, rounding
`ReformatOp`	`holoscan.vision.ReformatOp`	`input` → `output`	Interleaved or planar layout
`NonMaximumSuppressionOp`	`holoscan.vision.NonMaximumSuppressionOp`	`boxes` + `scores` → `mask`	Score and IoU thresholds

ResizeOp supports nearest-neighbor, linear, cubic, and area interpolation. RemapOp supports absolute, absolute-normalized, and relative-normalized maps. NormalizeOp keeps the input dtype. Use ConvertToOp when a U8 image needs to become F32 before normalization. BayerDemosaicOp reconstructs RGB or RGBA from a single-channel Bayer mosaic.

Image operators accept rank-3 HWC and rank-4 NHWC tensors, except BayerDemosaicOp, which accepts only rank-3 HWC. ReformatOp can also read and write CHW and NCHW. After it produces a planar tensor, the next Vision operator has to be another ReformatOp that converts back to HWC or NHWC. Otherwise the tensor should leave the Vision module.

NonMaximumSuppressionOp expects S16 boxes shaped [N, W, 4] and F32 scores shaped [N, W]. It writes a U8 mask of the same [N, W] shape. Boxes and scores for one call must share a capture time.

Use it in an application

Link the installed CMake package:

find_package(holoscan COMPONENTS vision REQUIRED)
target_link_libraries(my_pipeline PRIVATE holoscan::vision)

HOLOSCAN_BUILD_VISION is on by default. The Debian package is holoscan-vision, and it depends on the matching holoscan-core version. A source build can turn the module off with HOLOSCAN_BUILD_VISION=OFF. Python bindings are built when both HOLOSCAN_BUILD_PYTHON and HOLOSCAN_BUILD_VISION are on.

#include <holoscan/vision/resize_op.hpp>

auto resize = graph.op<holoscan::vision::ResizeOp>("resize");
resize.set_params(holoscan::vision::ResizeOpParams{
    .width = 1280,
    .height = 720,
    .max_batch_size = 4U,
    .interpolation = holoscan::vision::InterpolationMode::kLinear,
});

from holoscan_vision import operators as vision_ops

resize = vision_ops.ResizeOp(
    graph,
    "resize",
    max_batch_size=4,
    width=1280,
    height=720,
    interpolation="linear",
)

Each Vision output also needs an explicit device binding. Holoscan does not assume device zero.

options.deployment.bind_tensor_output_device(
    holoscan::TensorOutputDevicePlacement{
        .operator_path = "resize",
        .output_port = "output",
        .device = holoscan::DeviceId{0},
    });

Source is in the Holoscan SDK repository.

All modules