Archiving of Personal Diaries

[Pete Eeles]

Hi all,

As of a few days ago, the total number of files on the server hosting UKB was 344,615, far exceeding the allowable limit (based on the web hosting package) of 300,000. The vast majority of files are the result of images found in the Personal Diaries, where a thumbnail and separate larger image is created on the server for every photo attached.

The Personal Diaries are an incredible resource that should be retained for posterity, especially since they can be searched. And so I've improved the archiving mechanism whereby images are removed from older posts, while at the same time creating a PDF for each diary and year so that those images can, at least, be seen in some form. After archiving 2016, 2017 and 2018, almost 100,000 files have been removed from the server, and we're well below the 300,000 file limit. It's been a bit of a slog but will be worth it in the long run.

One enhancement is that each archived post will now show a message that emphasises that the post has been archived, while also providing a link to the relevant archive PDF. For example:

There is also a summary of all archived diaries for a give year at https://www.ukbutterflies.co.uk/forum_diaries.php, which is accessible from the Community menu on the main website.

I've also taken some time to update the entire 'ecosystem' with the latest versions of PHP, MySql and the forums now active. I've not really touched things for a few years so it was about time!

Any issues then do let me know!

Cheers,

- Pete

[Pete Eeles]

It's that time of year again!

Every year or so, it is necessary to prune the number of files on UKB to remain under a hard 300,000 file limit (this is an IT limit common among web hosting providers, which is a constraint imposed by the underlying operating system). Exceeding this number leads to the website being 'throttled', with poor performance resulting.

The personal diaries are the main reason that this limit is exceeded, and it's necessary to prune the diaries to ensure that we don't exceed the file limit.

To overcome this restriction, the personal diaries for a given member and year are archived and stored as a single PDF file (thereby allowing us to overcome the file limit) with all file attachments (usually photos) then deleted from the forums. If you see missing photos, then this is usually why.

Diaries for 2019 and 2020 have now been archived (prior years have already been archived), and has resulted in 76,294 files removed.

You can find the archives here: https://www.ukbutterflies.co.uk/forum_diaries.php

Of course, we could have just deleted the posts before a certain date, but the UKB team believes that there is too much valuable information being added to UKB as a resource, and that we should keep the forum entries (text) intact, and retain the text and images within an archive that is complete.

Any suggestions etc. are always welcome, but this is the approach being used for now (and has been for 10 years!).

Moving forward, we hope that an effective mechanism for linking posts, missing images and the archive will be in place.

Cheers,

- Pete