I don't have the sport rack and backrest, so I can't speak from experience, but looking at Firefly's excellent detail photos, it appears the front hooks and the latch on the sport rack plate are strong enough to handle an angular load. The added GIVI rack would provide additional stiffness and strength. The weak spot appears to be the mounting points. They would be strengthened substantially by the use of fender washers beneath the nuts, or better yet, fabricate a full metal support plate for underneath, and use longer screws.
-Scotty