Problems readying up in swf lobby

Morgeese

Been having this problem lately where my friend and I will both be in que together and one will get into the game lobby and one wont. Also when we return to the lobby after a match, sometimes we will both ready back up but on both our screens it shows that the other is not ready. So we leave that lobby and re invite each other and then join but are invisible like we arent in the same lobby? Are these common occurences or is our connection just bad? Usually resetting the game will fix it but its happening so often its getting annoying. Any suggestions or sure ways to fix? Playing on xbox btw.