What this site mostly does is select a window of a sparse subset of the comments table. That spares subset could be:
- all the comments on a thread - all the comments by a user - all the comments matching a search
It was getting painfully slow to view comments in a large result set because the URLs were using the ordinal number in the total result set to select which window of comments to show. Calculating those ordinal numbers required sorting every row in the comments table, which now has 1.3 million comments.
The new way is to use comment_ids in the URLs. That lets me select ranges, which is way faster. So now if you page through some thread or some user's comments, it should be 10x faster.
But working with comment_id's instead of "nth to mth comment in total result set" forced me to rework tons of code, resulting in unforeseen problems like the ones above.
patrick.net
An Antidote to Corporate Media
1,363,438 comments by 15,736 users - Al_Sharpton_for_President, Blue, ElYorsh, HeadSet online now