In case you were ever curious, it’s definitely possible to upload epic-sized files within the Plone CMS. Browser support aside (Firefox and your 2 GB limits, I’m looking you), I’d previously had issues with being able to upload files larger than 4 GB, if memory serves correctly. Previously, there was an issue with files causing out-of-memory issues, and I’m fairly sure the errors I was getting were related to what was fixed by this change.
For today’s test, Google Chrome was able to upload a 26.3 GB (28212643840 bytes) .tar file into Plone on my local machine:
It took a long while (even with my SSD) but it made it. No tracebacks or memory leaks with Plone/Zope so kudos to the developers. It’s interesting to note that the file upload process seemingly took 2x (double) the amount of disk space — half was for a file in the /tmp/ directory, and I didn’t end up tracking down the other one. Presumably something like pre-allocated space for the final file, perhaps.
Anyway, it works! Should you do this? Maybe, if you’ve got space and don’t mind hammering your CPU for your server during the upload process. I’ll go for larger next time..only problem is my SSD is only 240GB so something like a 500GB file (eg 1TB needed at least) will just have to wait.
Having a bad day? Just bombed your only copies of some files that you’d been storing as blobs together with your Plone database? Maybe you had the blobs stored incorrectly on your Zeo client, rather than the server, and then in a hasty effort to free some space (due to an on-going issue with blobs eating HDD space), deleted them. Whoops..they’re gone and their references are still in your Plone database. Or, in a less disastrous situation, maybe you just want to be able to run your Plone database without needing lost blobs? Read on.
Continue reading »
As the title suggests, there’s a conflict between a Zeo database instance being run together with a Plone 4/Zope instance, and them trying to share the same blob storage folder. But, this only happens if you misunderstand or incorrectly set the option of shared-blob = off, and blob-storage to be the same location as for Zeo in your buildout. If that previous set of (seemingly jumbled) thoughts doesn’t make any sense to you, then move along, nothing to see here.
Continue reading »
Another fun-and-games style problem I’ve come across when using plone.app.blob: sometimes migrations won’t work when converting a standard site’s files over to blobs.
That’s a pretty ambiguious description, but essentially, the error you might see will have a semi-normal traceback to start, and then garbage (contents of a file, presumably) – which, depending on the file size might hurt your browser. The last part of the (normal) traceback reads thus:
File "/home/buildout/instance/eggs/plone.app.linkintegrity-1.0.11-py2.4.egg/ plone/app/linkintegrity/handlers.py", line 158, in referencedObjectRemoved raise LinkIntegrityNotificationException, obj LinkIntegrityNotificationException
Thankfully, this gives us an excellent pointer to a solution: disable link integrity checking on your site prior to migrating.
The fix/workaround
Head to the Plone Control Panel, and click onto Site. Uncheck and then run your blob migration. When you’re done, you can return your site to normal. Or not. Your choice. Doens’t seem like there’s any harm with turning it back on once everything’s happy; just something to watch out when migrating content.