Software-defined
endoscopy
An FPGA design, Holoscan operators, and a sample image-processing pipeline for real-time display. Connect a PCIe-based FPGA capture card to GPU imaging and AI, then extend the pipeline for your device.
Early access under NDAAnnounced at GTC Berlin · General availability in 2027
Planned Apache 2.0 Licensed Open Source releaseDesigned to be inspected, modified, and extended
Image courtesy of CAMMA, IHU Strasbourg, and the University of Strasbourg
Start at the sensor
Connect MIPI camera streams to FPGA ingress, with a path into host or GPU memory for Holoscan processing.
Let the camera lead
Derive display timing from camera timing and set the processing budget for your imaging pipeline.
Keep a hardware path
Use an FPGA image signal processor to supply a live image when the host processing path is unavailable.
01 / CAMERA TO DISPLAY
One FPGA connects
input, output, and fallback
A PCIe-based FPGA capture card connects camera input and display output to the compute. Holoscan source and sink operators bring that I/O into your application graph, with a sample processing pipeline for real-time display.
GPU processing and compositing → framebuffer reader
A live image path independent of host processing
Holoscan processes the camera stream. The FPGA fetches the completed frames and transmits them on the camera-derived output clock.
Configure the camera
Adapt exposure, pixel format, frame geometry, and output delay through software configuration.
Process in Holoscan
Connect acquisition to CUDA image processing, model inference, and compositing in one application graph.
Return to the FPGA
Device-initiated frame reads put output timing under the FPGA’s control, with HDMI, DisplayPort, or SDI integration in the hardware design.
02 / A SHARED TIMEBASE
Camera timing
sets the cadence
Lock the output clock to the camera clock and choose a fixed processing budget between acquisition and display. The design avoids the extra buffering needed to reconcile independently running camera and display clocks.
Holoscan performs the image processing within that budget. The FPGA handles the scheduled frame read and line-by-line output.
03 / HARDWARE-MANAGED FALLBACK
Design for
a live fallback image
The reference architecture retains camera ingress, an image signal processor, and camera-clocked display output in the FPGA. That path can continue producing an image without the Linux and Holoscan processing path.
Monitor the output
Check frame freshness and buffer state to detect missing, repeated, or stale host output.
Select the hardware path
Switch image processing to the FPGA ISP while retaining the camera-to-display timing path.
Validate in your system
Exercise fault conditions, verify the transition, and integrate the behavior into your product’s requirements and verification plan.
Start with
early access
The Holoscan Endoscopy Reference Design is announced at GTC Berlin, with early access available under NDA and general availability planned for 2027.
Contact your NVIDIA representative to discuss NDA early access to the reference design and code samples.
04 / BUILD YOUR IMAGING PIPELINE
Bring your algorithms
into the image path
Explore the public Holoscan tool-tracking and segmentation examples today. These software building blocks provide a starting point while the complete endoscopy reference design is in NDA early access.
Endoscopy tool tracking
Explore video acquisition, inference, and tool overlays in Python and C++.
Tool segmentation
Explore the Holoscan application using a MONAI endoscopic tool-segmentation model.
Holoscan Modules
Explore reusable operators for camera input, image processing, inference, and visualization.
A starting point you can extend
The planned Apache 2.0 open-source release will let you replace processing stages, add your models, and adapt acquisition and output to your hardware. Explore the SDK and reusable operators as you plan your imaging system.