patrick.net

@zzyzzx OK, did it.

Now this is the definitive thread for the Pelosi attack:

https://patrick.net/post/1377463/2022-10-28-pelosi-s-husband-threatens-to-release

@patrick

I tried out tesseract today (an ocr lib) and it was easy to use. I was thinking maybe you could use it to make the text in images searchable. Tags still fill the void when the image doesn't contain any relevant text, but many images on here contain headlines, and so might be useful if they were searchable.

// npm install tesseract
// fyi I used node 18.12.0
const Tesseract = require('tesseract.js')
let filename = 'npr.png'

Tesseract.recognize(filename)
    .catch(err => console.error(err))
    .then(function (result) {
        console.log(result)
        console.log(result.data.text)
        process.exit(0)
    })

result of console.log(result.data.text)

m K@su L SIGNIN i NPR SHOP
[E] NEWS X CULTURE J MUSIC () PODCASTS & SHOWS  Q SEARCH >
TECHNOLOGY
4] . .
Elon Musk said Twitter wouldn't become
v a 'hellscape.’ It's already changing
E October 31}, 2022 - 4:'?9 PMET
‘v‘ gs Considered
fl SHANNON BOND

from



I think it worked pretty well out of the box w/ default settings! It still worked w/ my network cable unplugged, so they aren't calling some web spyware service to accomplish it - its done locally.

Maybe an implementation could be something like

CREATE TABLE image_words (
    post_id INT,
    file_name VARCHAR(255) NOT NULL,
    words VARCHAR(20000),
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP NOT NULL,
    most_recent_ocr_attempt DATETIME,
    most_recent_ocr_success DATETIME,
    FULLTEXT KEY (words),
    -- I'm not sure which of the following indexes would work best w/ mysql for the query, but one of them should work well I think.
    INDEX dates1 (most_recent_ocr_success, most_recent_ocr_attempt, created_at),
    INDEX dates2 (most_recent_ocr_attempt, created_at)
);

-- Have a background worker run this query and process the results every X minutes.

-- Find rows that have not been successfully processed yet. 
-- If an attempt was made on a row, but it failed, we try to process it again but not for at least a week. 
-- The reason we process newest first is a similar reason for why we wait 1 week before reattempting 
-- a failed row - this query+table is basically a queue, and we want to make sure we don't eventually clog the head of the queue up 
-- with stuff that keeps failing over and over, which might prevent the worker from ever consuming fresh work that it
-- can succeed with. Let it try the new stuff first, and then if it has spare time, it can reattempt failed stuff.
select * 
  from image_words 
 where most_recent_ocr_success is null
   and (
        most_recent_ocr_attempt < CURRENT_TIMESTAMP - interval 1 week
        or
        most_recent_ocr_attempt is null
   )
 order 
    by created_at desc
 limit 100

Hircus says

I tried out tesseract today (an ocr lib) and it was easy to use. I was thinking maybe you could use it to make the text in images searchable. Tags still fill the void when the image doesn't contain any relevant text, but many images on here contain headlines, and so might be useful if they were searchable.

I will try this. Thanks @Hircus

"npm install --save tesseract" failed:

npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! ../src/tesseract_bindings.cc:7: warning: "BUILDING_NODE_EXTENSION" redefined
npm ERR! 7 | #define BUILDING_NODE_EXTENSION
npm ERR! |
npm ERR! : note: this is the location of the previous definition
npm ERR! In file included from ../src/tesseract_bindings.cc:9:
npm ERR! ../src/tesseract_baseapi.h:10:10: fatal error: baseapi.h: No such file or directory
npm ERR! 10 | #include
npm ERR! | ^~~~~~~~~~~
npm ERR! compilation terminated.
npm ERR! make: * [tesseract_bindings.target.mk:125: Release/obj.target/tesseract_bindings/src/tesseract_bindings.o] Error 1
npm ERR! gyp ERR! build error
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack at ChildProcess.onExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:390:28)
npm ERR! gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
npm ERR! gyp ERR! System Linux 5.10.0-14-amd64
npm ERR! gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/patrick/webfam.net/server/node_modules/tesseract
npm ERR! gyp ERR! node -v v17.2.0
npm ERR! gyp ERR! node-gyp -v v8.4.0
npm ERR! gyp ERR! not ok

Could this have caused this?
https://patrick.net/post/1377561/2022-11-04-patrick-net-was-down-for-a-minute

Good suggestion. I didn't consider that.

Periodically, I'm unable to post replies, and receive an "Invalid Nonce" error message. If I try the better part of a day later, I'm able to eventually post my reply. Is this the result of an intrinsic or an external cause?

@stereotomy

It's because of a poorly thought out system for stopping XSS. Mostly it happens when people start to edit a post or comment, and then submit it more than two hours after they started it.

I will fix it so that never happens. Thanks for the reminder to do that.

Patrick do you have a way to aggregate all of the individual posts on Politicians, that you have been posting in the past year or so?
They could use their own section like Memes, Latest Images, ect. It would be nice to have a repository on the internet, that keeps the score of politicians as they rise through the ranks, then try to sell themselves as champions that their track record says otherwise. The media then hides or misrepresents those politician's track record, depending on if they are trying to bolster or destroy them.

Thanks @Tenpoundbass

I got a list of all 535 Congressmen and created a thread for each of them on May 31. You can add images, memes, or whatever to any of those as usual.

You can search for any of them by name, but you'd like some kind of search that lists them all at once, right?

It might be on my end, but I notice now I'm getting automaticly logged out every week or so. Not the biggest deal, more a minor inconvenience.

@GreaterNYCDude Huh, that should not happen and I have not noticed it myself.

Being logged in on patrick.net just means having a cookie in your browser that is an encrypted version of your password.

Sometimes people have their browser set to erase all cookies when they quit the browser. Could that be it?

@cisTits @GreaterNYCDude

Can you look at the cookie expiration date in your browser's dev tools? For me, it looks like this:



(Don't paste the actual value of your cookie here or others can see it and become you.)

So the expiration date for the cookie in my browser is about 6 months from now.

Patrick says

You can search for any of them by name, but you'd like some kind of search that lists them all at once, right?

Yes a searchable section dedicated to politicians, by office and party.
Perhaps on down the road, then I would like to be able make api calls to your Critter Catcher, you could become the go to source for such crowd sourced repository. Like the Craigslist, Wiki, or Facebook of who's who elected officials and would be candidates.

I would like to make a precinct map, and an aide for people to navigate their elected vermin vs heroes getting geo data for that is easy, getting info on the polls records on issues is not.

Ballotopedia has a lot of the facts:

https://ballotpedia.org/Rand_Paul

But they do not let you write comments about the politicians, I think.

Maybe they are deliberately erasing the patrick.net cookie. I don't trust DDG.

Patrick says

Ballotopedia has a lot of the facts:

Ballotopia is curated glorified facts for some, and regurgitated MSM smears for others. It's not a collection of posted articles with discussions as they happened. The media likes to rewrite history and play selective memory, it would be nice to have the receipts from when the episodes were unfolding before the media can spin and clean it.

Also with your critter catcher, people will post in them when RINOs vote with Dems on issues, that undermined our interests. Ballotopia only tracks bills they sponsored, not how they voted, or the detriment it had on their base at the time. People forget these events, it would be nice to have a place to replay the tape. We're looking for Backstabbing events, or Hero events, not general official records. Nobody remembers the ramifications of those when the next election cycle comes.

Tenpoundbass says

Yes a searchable section dedicated to politicians, by office and party.
Perhaps on down the road, then I would like to be able make api calls to your Critter Catcher, you could become the go to source for such crowd sourced repository. Like the Craigslist, Wiki, or Facebook of who's who elected officials and would be candidates.

How would this be different from search right now? If you know a Congressman's name and put it in, you will find a page for them.

I do plan to make an API so that people can use patrick.net programatically, or by using the API in an app (though I hate apps).

I have two suggestions:

1) I want the ability to tag a comment do I can refer back to it. Some things take days or weeks to research. I can bookmark them (and do), but it would be nice to have the ability to tag/untag a comment

2) It would be nice to have a list of responses to what I've written. I know this is sent in email, but I think it would be better to have a pull down on responses you need to take note of.

richwicks says

I have two suggestions:

1) I want the ability to tag a comment do I can refer back to it. Some things take days or weeks to research. I can bookmark them (and do), but it would be nice to have the ability to tag/untag a comment

2) It would be nice to have a list of responses to what I've written. I know this is sent in email, but I think it would be better to have a pull down on responses you need to take note of.

like favorites? or just searchable tags?

FortwayeAsFuckJoeBiden says

richwicks says

I have two suggestions:

1) I want the ability to tag a comment do I can refer back to it. Some things take days or weeks to research. I can bookmark them (and do), but it would be nice to have the ability to tag/untag a comment

2) It would be nice to have a list of responses to what I've written. I know this is sent in email, but I think it would be better to have a pull down on responses you need to take note of.

like favorites? or just searchable tags?

Searchable tags would be nice, but I think that's overkill. Say you can tag a comment with a name, and search based on that..

I'm thinking more like a website favorites.

Tenpoundbass says

Yes a searchable section dedicated to politicians, by office and party.

@Tenpoundbass OK, how about this:

https://patrick.net/post/1377838/2022-11-29-us-congressmen-lists

richwicks says

Searchable tags would be nice, but I think that's overkill. Say you can tag a comment with a name, and search based on that..

Well, every comment already has a unique id number, and you can find any comment by number like this:

https://patrick.net/comment?comment_id=1902282

Or did you mean a way to give multiple comments the same tag?

Patrick says

Tenpoundbass says

Yes a searchable section dedicated to politicians, by office and party.

Tenpoundbass OK, how about this:

https://patrick.net/post/1377838/2022-11-29-us-congressmen-lists

I think just having a checkmark to the left to tag a post as something you need to follow up on would be nice.

I imagine having a button to the LEFT of a post might be difficult to implement but something like that would be ideal.

Basically, bookmarks for the site is what I suggest.

@richwicks OK, something like the existing "pinned" list for threads:

https://patrick.net/pinned?a=Patrick

But for comments instead, right?

It wouldn't say why you pinned a comment, but hopefully you'd remember why.

Patrick says

richwicks OK, something like the existing "pinned" list for threads:

https://patrick.net/pinned?a=Patrick

But for comments instead, right?

It wouldn't say why you pinned a comment, but hopefully you'd remember why.

That would be good enough. I mean, I just need to know "I promised to respond to this" - but it takes more than 5 minutes for me to do it. One I have to do, is read a law in Pennsylvania that legalized mail in voting.

OK, it's kind of a good time for this, since I was planning to merge the database tables for original threads and comments so that search could be unified. Right now there are separate tables and it's a pain.

After they are unified, then there can be just one kind of search, and one kind of pinning.

Patrick says

OK, it's kind of a good time for this, since I was planning to merge the database tables for original threads and comments so that search could be unified. Right now there are separate tables and it's a pain.

After they are unified, then there can be just one kind of search, and one kind of pinning.

Can I make a suggestion?

Make a PARALLEL system to start with if you're about to do an overhaul. If the underlying database isn't changing, you can run two interfaces at the same time on top of the DB.

Also - BACKUP.

I do have a reasonable daily backup system I think. Worst case, one day is lost. Eventually I hope to have all comments immediately mirrored to a different server, but that's time, money, etc.

For the db migration, I plan to take every thread, make the text of it a comment, and have that thread just be a kind of skeleton which just has a bit of metadata and points to the original post content in the comments table.

So everything is going to be a comment. A "thread" will just be a wrapper around the first comment, which is the original post.

The whole site should look and work pretty much the same, except that search and pinning will be just one thing instead of two.

Patrick says

I do have a reasonable daily backup system I think. Worst case, one day is lost. Eventually I hope to have all comments immediately mirrored to a different server, but that's time, money, etc.

For the db migration, I plan to take every thread, make the text of it a comment, and have that thread just be a kind of skeleton which just has a bit of metadata and points to the original post content in the comments table.

So everything is going to be a comment. A "thread" will just be a wrapper around the first comment, which is the original post.

The whole site should look and work pretty much the same, except that search and pinning will be just one thing instead of two.

Again if you like (and you probably don't), I can help you clone this machine and you can run it locally for development. Setting up a website in your home is easy, but the name of the site will change, and you'll need to open ports on your router, and to setup let's encrypt. I can show you how to do all that.

Thanks, but I've got this, no problem.

Patrick says

Thanks, but I've got this, no problem.

Well, I can see you're using Let's Encrypt still and have the script running for it.

Setting up your router to allow 80 and 443 passthrough is all you need to do to test locally at home - and REALLY you don't need that OR Let's Encrypt unless you plan to allow access by the wider world.

All you need to do is clone the virtual machine. To do that, setup a VM that is CLOSE to what you have here, and then use sudo apt --installed to get a list of packages you need and/or are missing.

SMTP won't work though, that's a BITCH to setup on anything other than an "approved" server.

I reformatted thread lists (home page, author list, search results) to always have the user icon in the thread summary.

Also, the first image is much larger, and more of the text is included.

Noticed the HUGE IMAGES and I liked it better the other way. I have to scroll a lot more to read the next set of thread titles. Compact is better!

OK, I'll make them smaller.

@rocketjoe79

How about now?

Heck, I just removed images from post lists entirely, except for the author icons.

Cleaner, but then the site looks like mostly text.

Actually, all of those changes sucked.

Reverted everything, so site is back how it was.

Looks great again!