Could be both, only way would be to try that.
First questions would be about plugin version you have, and checking if all the links are set correctly.
Often I saw users mistakenly mixing www and non-www versions of their links thinking it's the same domain, while technically they're different and so removing in one might result in NOT removing in the other because of domain-based sessions.
Same goes for SSL vc non-SSL versions of the site.
So I guess the 1st question is if all links are set correctly. Which URL you have when visiting the cart?
And when you click the remove button, are you still in the same domain scheme?