Was observing high CPU usage and memory leak after upgrading to Debian testing. And... lucky me, I found this bug report. Read through and decided to follow what others have done.
I removed the directories ~/.cache/tracker and ~/.local/share/tracker. My computer was back to normal :)