UKB limits reached!

[Pete Eeles]

Hi folks,

With the forums running for as long as they have, we've now hit a limit with the service provider since we have over 200,000 files on the server ... mostly images added as attachments to posts. This needs to be reduced before 21st August, otherwise UKB will be taken down. Here are some possibilities:

1. There is a mechanism for "pruning" old posts (which will remove the post and any attachments) where you can select the number of days to retain (which takes into account not only posting, but also viewing). This would, for example, leave all personal diaries intact if I were to specify a numbers of days that equates to, for example, 5 years.

2. Another option is to clean out specific forums, such as the identification forum - which generally contains one-off requests that are never perused after the items have been identified.

3. A further option is to delete any attachments associated with posts before a certain date (e.g. only keep attachments associated with posts made in the last 5 years). The trouble is, personal diary entries may see images disappear from early posts.

Unfortunately, there is no option available to retain posts, but delete attachments, before a certain date.

Before I do anything, I want to debate this on UKB. I'd love to at least keep the posts that have been made (even if the attachments are lost), but am struggling to see how to do this at the moment.

Comments most welcome!

Cheers,

- Pete

[Goldie M]

Hi! Pete, I think the third option is good, having said that how many of us go back to what we put in five years ago.
Most people are more interested in the here and now and if they down load their details and all their photo's has I do onto a USB then nothing is really lost Goldie

[Padfield]

This is a great shame - I often do refer to very old posts. Is this an absolute limit or a function of the amount you pay for the site to be hosted? If the latter, how many members would have to pay a subscription of how much to access older posts and attachments? I'm just thinking on the hoof here, and it might be completely stupid - forgive me. My nebulous idea is that free membership would entitle you to search and view all posts up to, say, a year old. More committed members, who keep a running diary and use the site to record their own butterfly life, might want to purchase a yearly or lifetime subscription, which in turn would enable you (Pete) to purchase a larger, or preferably unlimited, allowance.

I think it is very important for UK Butterflies to remain free but I for one would pay a little somethin to keep it complete.

Guy

[Pete Eeles]

Thanks for the considered response both. I agree Guy - it would be a real shame to lose any information whatsoever!

I've been in touch with JustHost (my provider) who, I thought, provided unlimited bandwidth (data transfer) and storage. It seems that, in the fine print of the package I have for UKB, there is a limit of 200,000 files.

There is an option to upgrade my subscription package for an additional £12/month. This would probably last us a few more years before we hit this package's limit, which is set to 300,000 files for this package. So this would simply delay the problem but might be a good solution in the short term.

There is also an option to move to a dedicated server, which would cost £65/month. This has no limits in terms of number of files (but is limited by 240 GB of storage).

Cheers,

- Pete

[David M]

Like Guy, I too would be prepared to pay a 'fee' to retain this forum's integrity.

Perhaps an invitation to donate would work rather than a general subscription which would undoubtedly deter those members who post infrequently as well as potential new recruits!!

I really don't like the idea of 'culling' the early posts from personal diaries. I prefer the option of deleting ID requests from 'way back' but this would only be a temporary sticking plaster.

How much per annum needs to be raised to increase bandwidth? £100? £200? I'd gladly go to £20 per year if it would guarantee the preservation of the images contained in personal diaries.

[Vince Massimo]

This is a bit unexpected

I see that the forums started in 2006 but, when looking back at the older posts, many of the associated images are now not available for various reasons. This is just an observation.

As the problem mainly seems to relate to images attached to posts, I assume that images in the Species-Species Albums are not being considered for pruning.

If there is no choice and we clearly have a deadline, then I would favour option 2, but I fear that this may not free up enough capacity and we may have to resort to option 1 as well. If a 5 year cut-off date is not introduced then the problem will continue to occur.

I may have some further thoughts in due course, but would be happy to pay a subsrciption.

Vince

[Pete Eeles]

Thanks chaps, much appreciated. Once this thread concludes, I'll see if I need to raise any funds, but £65 per month I cannot do! Perhaps a major sponsor would be the way to go, if anyone has any ideas.

Option 2 would give us a few years and enough time to get things sorted out properly - although I'm not sure (at this time) what "sorted out properly" actually means! It would also give me enough time to explore the forum mechanism as well to see if we should consider moving away from the current implementation (for various reasons that I won't go into here!). I'll also be looking for another IT-savvy team member to help in the background with the mechanics of the website.

It was a surprise to me as well, Vince, since I chose JustHost on the basis that storage was unlimited, which it clearly isn't!

Cheers,

- Pete

[Pete Eeles]

When looking back at the older posts, many of the associated images are now not available for various reasons.

I'll have to look into that.

As the problem mainly seems to relate to images attached to posts, I assume that images in the Species-Species Albums are not being considered for pruning.

Absolutely not. They, and personal diaries, will remain intact whatever happens. The problem is with files in general, but those resulting from attachments far outweigh anything else.

Cheers,

- Pete

[David M]

Perhaps a major sponsor would be the way to go, if anyone has any ideas.

Given that this site has an enviable inventory of images which may well at some point be required for use by any number of organisations, would it not be possible to allow those organisations to use these images as and when in exchange for some kind of regular (and very insignificant) financial support?

[Pete Eeles]

Perhaps a major sponsor would be the way to go, if anyone has any ideas.

Given that this site has an enviable inventory of images which may well at some point be required for use by any number of organisations, would it not be possible to allow those organisations to use these images as and when in exchange for some kind of regular (and very insignificant) financial support?

That's certainly an option, David, and we do contribute images (free of charge) to charities, academia and other non-profit entities and initiatives. However, I wouldn't want any conservation monies to be diverted, personally. I'm much rather we worked with an organisation that is looking to demonstrate its green credentials, much like BC has with M&S.

Cheers,

- Pete

[David M]

Well, perhaps it's time we approached M&S to see if they are interested in assisting what is, easily, the biggest and most popular butterfly website in the whole of the British Isles.

How many people on here are active in the 'Big Butterfly Count'? Let's be honest, this isn't a one-off vanity project; it's been going for a few years now and shows no sign of having its momentum curtailed.

From a personal viewpoint, I'm far more inclined to see Marks & Spencer in a positive light as a result of their continued involvement in this initiative.

We may be relatively small in number, but I think the vast majority of us are extremely grateful that organisations carrying such kudos continue to support butterfly conservation.

The cost to them would be utterly minuscule, and they could benefit from free advertising on a site which, I trust, is subscribed to by several dozen folks who are part of the very sector of society that they wish to target.

It surely can't hurt to pose the question.

[MikeOxon]

200,000 files sounds a remarkably small number, or am I misunderstanding? If they were 1MB each, on average, that's only 200GB, which could easily be stored on a 'small' hard disk. If old attachments could be stripped off the on-line servers then, if an individual needed access to a specific attachment, they could e-mail a request for the specific item from the disk archive - a small fee could be charged for this service. Some other blogs I know have 'archive' versions that can be accessed but not modified in any way. There is also software around that can download an entire website.

I find it hard to believe that there is much of real significance in the old threads that hasn't already been picked up for the species-specific albums. I would also assume that, in most cases, the original posters have copies of their attachments, which could be requested from them if the attachments were deleted. In my own case, I keep PDFs of my personal diary on an annual basis, which are much easier for me to look back through than the actual posts.

This may seem a bit of a shock, coming 'out of the blue' but is it really a significant issue? If the current provider has these terms, perhaps it is time to consider a move anyway.

[Pete Eeles]

Panic over - at least for a while! I've just upgraded to the next package up ("Business Pro") until 2018, which will increase the limit on the number of files to 300,000. I'll now be keeping a close eye on how many files we have and now have enough time to get the file issue sorted. I like the idea of archiving items that haven't been viewed for some time, with a mechanism for bringing these back online "on demand" as it were - but that will take some investigation and implementation.

Also - since there have been some generous comments made in thread (for which I'm truly grateful) - the new package provides a number of add-ons (free) that I previously paid for. With the (effective) cancellation of my previous hosting package, the cost of the upgrade came to a whopping £42

As for hosting providers that scream "UNLIMITED STORAGE" at you ... don't believe them. Just Google "200,000 inode limit" and see for yourself!

Also - since the migration is happening right now, there may be some disruption to UKB while this completes (I assume all files are being moved to a new server).

Cheers,

- Pete