Purging Obsolete Media Files

TiG
TiG
@tig
10 months ago
182 posts
We have a need to purge obsolete space-consuming media files. Our plan is to identify comments (and articles) that are no longer considered 'timely' and remove all image, video, etc. files associated with those comments (and articles). The embedded
img src="..."
, ... HTML will then be removed from the comment (or article).

In result, these obsolete comments (and articles) will be stripped of their associated HTML media references and the stored media files will be purged to free up disk space.

Key question is which JR functions should be used support this algorithm? How, for example, can we best translate the
img src
information into a reference to the stored media file and then, given that reference, what is the best JR function to use to delete just that file?

Thanks.


--
TiG

updated by @tig: 09 Nov 2021 11:34:47PM
michael
@michael
9 months ago
7,496 posts
"that file" is the hard part. All images that come out on "img src" will first be stored in the cache at /data/cache/(module name).

They will have been put there by processing the master image which is stored with the data at /data/media/(one number)/(another number)/jrAudio_2_image.jpg

something like that ^

That's the image that needs deleting but if you just delete it without deleting the associated datastore entries the system will think it still exists.

So for an image like this one:
your-site.com/blog/image/blog_image/2482/xxxlarge/_v=63f541628547989

its a blog image, its datastore key is 'blog_image' and the datastore _item_id is 2482

Functions that may be useful:
jrCore_get_media_directory
jrCore_get_media_file_url
jrCore_delete_media_file
jrCore_delete_item
TiG
TiG
@tig
9 months ago
182 posts
Michael

My prior analysis led me to the media functions (among others). Your response has raised my confidence that these are the functions I should be using.

It does seem that there are quite a few combinations of media file names. I see variations like:
 src="http://thenewstalkerscom.jamroomhosting.com/data/media/0/0/Ning_Media/discussions/0/101_discussions.jpg" 
and
src="http://thenewstalkers.com/upimg/image/upimg_file/4852/256" 

That is why I was crossing my fingers that there was a 'magical' function that could resolve the variations into a single file reference that could be passed to:
jrCore_delete_media_file($profile_id, $file)
or
jrCore_delete_item_media_file
or
jrCore_confirm_media_file_is_local
.

Given your lead, I will use the media module as my epicenter and see what I can discover from there.

Thanks again for your insight,
TiG


--
TiG

updated by @tig: 09 Aug 2021 07:18:51PM
michael
@michael
9 months ago
7,496 posts
maybe follow this flow:
Upload a bunch of images to a single jrGallery then it will have a DELETE button for each. That would setup a structure where you can follow the flow of functions that work to delete a single file from a datastore item that contains multiple images.

Im guessing thats what you're after rather than deleting the datastore item itself.

--edit--
the path in the image should be /index.php not /image.php, sorry.
a-delete-flow.jpg
a-delete-flow.jpg  •  1.4MB


updated by @michael: 09 Aug 2021 08:02:47PM
TiG
TiG
@tig
9 months ago
182 posts
Michael:

There are a number of massive files (via FileZilla) that we know we can delete to free up space. These are video files. I must ask if it is possible to literally delete the files. What is the effect on the balance of Jamroom if a video file is deleted? My test shows that the embed command in a comment for a deleted video simply returns a null string (which is perfect).;

Do you see any downside to us going directly to known obsolete media files and simply deleting them from the file system and not taking any other action?

Thanks,
TiG


--
TiG

updated by @tig: 10 Aug 2021 05:37:02PM
michael
@michael
9 months ago
7,496 posts
Worst case I can think of: the user sees an error that is meant for the developers and doesn't understand it. A file not being there when the database expects it to be is not going to topple the system or crash it.

Maybe the item still appears in a list of items then when clicked on it goes to a 404 not found page. You can log 404 errors to the activity log via a setting in the ACP if you want to watch for that.

I often download a database of a site that needs working on from production but do not download the /data/ folder. The result is the site still works but the images show the default image placeholders.
TiG
TiG
@tig
9 months ago
182 posts
Michael

That is what I figured. We might do a quick trim of a few old, huge files just to give us some breathing space while I develop an automated solution to more gracefully manage obsolete wasted storage.

Thanks again,
TiG


--
TiG
michael
@michael
9 months ago
7,496 posts
if you're looking to expand the storage size but not the server capabilities we do have facility to add extended storage to your current hosting account. The advantages are bigger storage for the server without the price jump of an actual larger server.

Brian's set it up for a couple of people with large space requirements but not such a big demand on traffic.

Have a chat with him if that's something that could be useful in your case.

Tags