Hi There,
One of the customer is using an open-source tracker [link] that does not handle randomly appearing bounding boxes from model predictions effectively and bounding boxes written directly from the streamer pipeline.
Upon analyzing frames from the recorded video, we observed a lag in bounding box updates, where they are refreshed only once in every two consecutive frames. This behavior also impacts the tracker’s performance.
fyr, here is pipeline used by customet
gst-launch-1.0 v4l2src name=src device=/dev/video59 io-mode=dmabuf ! video/x-raw,width=960,height=540,format=RGB ! tee name=t_raw t_raw. ! queue leaky=2 max-size-buffers=2 ! glvideomixer name=mix sink_0::zorder=1 sink_1::zorder=2 latency=100000000 ! fpsdisplaysink text-overlay=true sync=false video-sink=“glimagesink fullscreen=0 sync=false qos=false” t_raw. ! queue leaky=2 max-size-buffers=2 ! glupload ! glcolorconvert ! video/x-raw(memory:GLMemory),format=RGBA ! glcolorscale ! video/x-raw(memory:GLMemory),width=640,height=640 ! glcolorconvert ! video/x-raw(memory:GLMemory),format=RGB ! gldownload ! tensor_converter ! tensor_filter latency=0 framework=neuronsdk name=nn model=/usr/bin/nnstreamer-demo/yolov8m_full_integer_quant_640.dla inputtype=uint8 input=3:640:640:1 outputtype=uint8 output=8400:84:1 ! other/tensors,num_tensors=1,types=uint8,dimensions=8400:84:1:1,format=static ! tensor_transform mode=arithmetic option=typecast:float32,add:-3,mul:0.0039636665023863316 ! tensor_transform mode=transpose option=1:0:2:3 ! tensor_decoder mode=bounding_boxes option1=yolov8 option2=/usr/bin/nnstreamer-demo/coco.txt option3=0 option4=960:540 option5=640:640 option6=1 ! mix. &
Regards
Sathish Kannan B