I haven’t made any significant progress with this for over a week now… it is really frustrating.
I think for the time being, I’ll revert back to one of my older algorithms that simply says whether a website was significantly updated or not. I am going to stop worrying about displaying the updated content for the time being.
So now, I can focus on designing the site (and I’ll keep in mind that at one point or another I’ll want to be displaying the updated content, so I’ll design accordingly). Finally maybe I can get some actual work done on this thing, and people can see what I’ve been ‘wasting’ my time on for the past month.