I was actually rooting for them to stay together and the way he just watched her walk away made my heart ache