I had this self same problem, and I found the answer elsewhere.
Go into settings and change the tv from Fast Start/Standby, and put it so as the TV actually turns OF when you hit the power button. This will clear the memory that is caching and using up the small amount of memory available.
It might mean that you have to turn off the TV during use, but at least it solves your problem for now.