Creating custom Queue and cloud module
Using Jamroom
DannyA:So how does the function on the remote server, or even on the main app server, know to look at the queue for a job?
It does not need to - the core handles it.
Quote: And how do you set it to get a file from s3.
If you are using the S3 module, and it is the active media system, you can just use jrCore_media_read_file(), which will "read" a file from S3 and bring it over.
Quote:Let me give you the specific use case.
I have a CLI for a waveform creation tool. I want the waveform creation to be performed by one of my conversion worker servers.
1. I assume the bpm detection module has to be on the conversion server.
2. I assume, when a file is finished uploading, I create a new queue task " create waveform".
Yes to both...
Quote:
3. How does the waveform creation module know to look at the queue for a new job?
It doesn't:
- when the queue is CREATED, it will be created on the same server OR on the queue server (if you have configured a queue server);
- when the core on the server that has the queue entry "sees" there is a queue entry, it will fire up the right function.
So you don't need to worry about that.
Quote:4. How does the waveform creation module know to look in s3 for a specific file instead of local file system
It doesn't need to - as long as all the file reading/writing/checking is done using the jrCore media functions - i.e.:
if (jrCore_medie_file_exists(1, 'MyModule_1_file.mp3')) {
$_item = jrCore_db_get_item('MyModule', 1);
jrCore_confirm_media_file_is_local(1, 'MyModule_1_file.mp3', $_item, '/tmp/file.mp3');
}
So what this does is:
- checks to see if the file 'MyModule_1_file.mp3' exists for profile_id 1
- if it DOES exists, it ensures there is a local copy of it at /tmp/file.mp3
You don't have to care what file system the file is actually on.
When done, you can "write" the file back:
jrCore_write_media_file(1, 'MyModule_1_alternate_file.mp3', '/tmp/file.mp3');
This would write the /tmp/file.mp3 file out to the proper location (and call it MyModule_1_alternate_file.mp3) regardless if you are using the local file system, s3, whatever.
The idea here is that as long as you avoid using function calls like is_file(), file_exists(), etc. (basically functions that only work with a local file) you don't have to care - the jrCore media functions will "wrap" everything and basically abstract the filesystem away.
Let me know if that helps.
