Ok, So I have searched the site for solutions, and the problem in the related topic seems to match my issue, however, I do not want to hijack the thread with my own issue. I have attempted the solution in the related topic, but the offending Update (KB33002567) is not installed on either the main server or our RDS Gateway/Broker 99% of users are connecting with no issue, but some users are getting stuck on "Securing Session"
This is on Server 2012 Datacentre with Windows 8.1 Virtual Desktops.
I have also tried the solution in this question but it still results in "securing sesssion"
My event log shows errors like this
Remote Desktop Connection Broker Client failed while getting redirection packet from Connection Broker.
User : Domain\User.name
Error: Element not found.
and
Remote Desktop Connection Broker Client failed to redirect the user Domain\User.name.
Error: NULL
EDIT: As an update, since I wrote this question, all users are now logged in. But it still took quite a bit of effort