49
posted ago by jackwerntz ago by jackwerntz +49 / -0

1071403 posts recorded in the database, to be exact.

These are exactly what I say they are. Three zip files. There are no executables. The HTML pages contain no scripting. The SQLite database file is just that. The SQLite database file will not be useful to most people, but can be accessed and read using a utility like DB Browser for SQLite, which is free.

Please understand that I could not get everything. I did not have the capacity for the linked images, videos, et cetera. I would run my python scripts a few times a day to process HOT, RISING and sometimes NEW when I was on shill duty. I also had a version that created HTML pages of my searches, and I did a lot of different keyword searches. I did not download linked content with the scripts. The SQLite database just contains the traces of a million plus posts. If they were text posts, the text is in the SQLite database and in the HTML files.

After I got this up and running, I modified it to get a varying percentage of top comments on each thread as it was processing. These comments are not in the SQLite database file, but show on the thread entries in the HTML files. I didn't start processing comments on their own until later in the process, so I am not including the comments database.

Big folder of HTML pages generated from posts https://mega.nz/file/omxxDKYR#6xBjYCmxS6WiosLlalZICtj-quzzdRGmhMFit0ETYXc

SQLite Database file https://mega.nz/file/A2hymIqY#Oic-u4IVJAAfesEkCj4944qH-Gd2glmPMqFVAkalXok

Tiny CSS file https://mega.nz/file/RrwyWQzK#kdO-e8NYWMO3o7QqurmsIBYrWokoQuVwadP0Zl7SA10

Unzip the big folder of HTML pages, make a folder called 'styles', unzip the style file there. The big folder of HTML files and the styles folder should be on the same level, as the HTML files look back up one directory level for the style folder.

We existed. It was real. These are just traces of what was, but it's what I could do.

Comments (6)
sorted by:
You're viewing a single comment thread. View all comments, or full comment thread.
5
thxpk 5 points ago +5 / -0

Great work pede