Media storage and caching

Brian, you mentioned S3 support, maybe as part of a media abstraction layer, was in the works. You mentioned you could try to have it in June. Is that close?

I'm running Jamroom on EC2 and really trying to do a couple of things:

1. Store media assets in S3
2. Leverage cloudfront for improved delivery of all cached assets.

I know you think that the jr caching is enough, but Cloudfront is no more expensive than regular delivery while taking the request off the server and delivering closer to user. It would be stupid NOT to use it when it would be a simple re-write of cacheable url's.

Can you let me know the status of these?
updated by @dannya: 07/13/14 05:28:50PM

If your wanting to reduce delivery time of cached pages without need for reprogramming of the system take a look at cloudflare.com

The integration is similar for cloudfront. All you are doing is pointing the static and cacheable assets to a different host name. Secured assets would use another host. So really the module just needs to do a re-write of the url for defined directories and file types. I'm just not sure which one those are.

Also, using a different CDN on Amazon would lead to additional outbound bandwidth charges.

That was my original goal - the 5.2 work took longer than expected, so that's going to be tight - I am working on it, but am running into some issues getting some things to work, so I don't want to guarantee anything until I know for sure it will be released, but work continues.

Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

I'm not sure why you think it is just a simple rewrite of URLs.

1) When JS and CSS are rebuilt how to do they get uploaded to S3 to be made available to the CDN?
2) Once uploaded, we have to tell the CDN that a change has been made (i.e. refresh)
3) same applies for any skin or custom image

This is not going to be supported any time soon - you would want to do something custom if this is a requirement of yours. The only work I am currently working on regards to S3 is media storage only.

Sorry!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
updated by @brian: 06/12/14 11:33:37AM

Yes, all CDN's work that way.
1. The cdn grabs content from the origin, wheather S3 or anywhere else, through a reverse proxy.
So any object you want delivered through the cdn you deliver through different host.

for example.
i create host cdn.mysite.com. IN the cdn control panel, you define your origin as www.mysite.com.
Now lets say you have an image on the site, such as logo. You would make the request for loge http://cdn.mysite.com/skin/elastic/logo.jpg.
The logo would be deliverred by the CDN. The FIRST time the request is made for the object, it is not on the CDN. So the CDN will redirect the client to http://www.mysite.com/skin/elastic/logo.jpg . HOwever, now the cdn has the object and can deliver it for future requests as long as it remains on the cdn network somewhere. This leads to the second question :refresh.

This works the same way it does on the local cache. There are a number of things to do.

1. You can define the TTL of your objects in the CDN control panel. You can create rules for different object types and directories.
2. All CDN's allow you to do a cache flush and immediately expire all your content so that the cdn must re-fetch all the content. This can be done in the console. However, most also have API's that allow you to trigger the flush from the app. In our case you could add a web hook that triggers a flush on CDN whenever you flush cache in JR.

I've been doing this for 14 years. It really is very easy and most of my cdn customers implement it in under an hour.

That being said, can you give an ETA on the S3 module?

I've done this many times as well, but thanks for the primer. I would not release a module for JR that required the user to "manually" do anything, which is why I'm not working on it right now - that's more involved then needs to be at this time.

No ETA on the S3 module - hopefully sometime soon but I don't want to give you a date and then tell you it can't be done because I ran into a road block.

Sorry!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net