Not sure if it would fix your issue, but you could try setting the
"do-timestamp" property of ksvideosrc to true, or by adding a
videorate element after your source. In my experience the video
rate element is preferable, because some elements (like
dshowvideosrc) occasionally don't timestamp frames for whatever
reason, which can cause problems with certain pipelines.
With the second issue, do you still run into this even after you
remove the autovideosink branch on the "server side" pipeline?