Forum Activity for @brian

s3 and conversion server

Jamroom Developers

That's cool - if you're going with AWS that's no big deal - I don't share your perspective that they are more expensive though - you get what you pay for.

The $20 a month 2 CPU linode will run circles around the 2 Gig EC2 instance (which has a single virtual CPU equivalent to a circa 2007 1.7 GHz AMD athlon). I've personally built and scaled 2 large systems on AWS and won't do it again - you end up having to run 50 front ends on their small/medium instance size when I could run the same on 5 linodes. And don't get me started on the inconsistencies of EBS (it can't compare with Linodes 1GPs SSD's).

I'm also not a fan of their reliability (look at their recent half day cloud front outage), but I know some people feel that that is a strength of AWS, so that's just my opinion.

If you're doing big bulk transfers however, the bandwidth is definitely better.

Regardless, I know many of our customers are going to choose AWS so we need to make sure Jamroom works awesome on AWS - we just may not be able to support everything AWS offers as a module in JR.

When we take the wraps off the cloud setup you can check it out and let us know what you think - that would be great.

Thanks!

Editor and Sharing...

Using Jamroom

I will check this out - the code should be stripped before being shared.

Thanks!

Smiley Support

Using Jamroom

You can use smileys anywhere on the site - not just in the editor - i.e.

s3 and conversion server

Jamroom Developers

I think a big part comes down to what type of traffic do you anticipate? I know it's fun to think you'll have 10 million visitors from day one, but even the "hottest" SV startups rarely get that type of growth in their first year.

A realistic scenario (based on my experiences with several SV startups and Jamroom for 11+ years) is that you'll get a few dozens to a few hundreds of visitors per day for the first few months to a year. Get some exposure on something like hacker news, Reddit, etc. and you'll have a "spike" to maybe 5,000 for a day (it tails off really fast).

So if I was starting a site, based on what I know of your project, here is what type of system I would setup for launch (all on Linode):

- (1) 2 Gig VPS app server (Ubuntu 14.04, Apache 2.4, PHP 5.5) running Jamroom + Cloud modules
- (1) 2 Gig MySQL Server - runs MySQL only (no Jamroom)
- (1) 2 Gig Conversion Server - runs Jamroom with Cloud modules + Cloud skin

Enable backups on the MySQL server - the other ones can be rebuilt from an image if needed. All connected via private networking.

Your scaling plan:

- Upgrade your app and conversion servers to accommodate more traffic (i.e. 4Gig, 8Gig, etc.).

- Keep an eye on MySQL load on your MySQL server and upgrade to larger VPS as needed.

- Add 1 or more MySQL cache and Session servers (part of the cloud modules) to offload cache and sessions from your main MySQL instance (which is a big hitter as your system gets busier)

- Add a Log Server for centralized logging (part of the cloud modules).

I know this plan does not scale the app server horizontally (which can be done, but then we need to use extremeFS - doable but I'm trying to keep it simple).

You can started for $60 per month and easily handle tens of thousands of visitors a day. You'd have 9 terabytes of transfer included before you paid anything extra for bandwidth.

It's also a MUCH simpler system to administer and manage over AWS, and can be launched right now to prove your MVP.

So anyways, that's what I would do if I were doing it

Hope this helps!

p.s. when we take the Jamroom Cloud profile public we'll have more detailed info on recommended cluster setups, including scaling using extreme FS.
updated by @brian: 12/02/14 11:07:06AM

s3 and conversion server

Jamroom Developers

Just to add a follow up (we may have already talked about this in another post), but have you considered saving yourself a ton of money and moving off of AWS?

s3 and conversion server

Jamroom Developers


I mean the unfinished work I started on an S3 module. The conversion server is already running. It does not however store anything to S3.


Fuse allows you to "mount" S3 to your EC2 instances as if it were another hard drive on the EC2 instance - i.e. you have 10 EC2 instances you could mount a "media" partition that is really a "media" S3 bucket. This allows PHP to read/write as if it was a regular file system.

The problem with the S3 API is just that - it's an API and not a file system. It works very differently than a regular file system and has many different shortcomings and caveats that you have to think about when using it. I don't want a module developer to have to worry if their module is on a Jamroom site running on S3 or a local file system - it should work the same. Unfortunately S3 and file systems do not work the same, so that's very hard to do.
updated by @brian: 12/02/14 10:12:25AM

question for paul re specific database tables i need to clear

Installation and Configuration


Just to add that if you're doing an import to a new Jamroom site, you'll only see the members import option until more than 1 member exists in the site, so it tries to help you there
updated by @brian: 12/29/14 09:23:54AM

s3 and conversion server

Jamroom Developers


I'm not in the office this week so really cannot get it into a state where it can be shared. I'm also hesitant to "release" something that I can't support, but I'll see what I can do when I get back.

I have been working on a set of "Cloud" modules for Jamroom that I think you'll find useful, but they may not work exactly for how you want to set things up. I think ultimately we are going to promote using your own cluster filesystem over S3 - i.e.

http://www.xtreemfs.org/index.php

As that just has a lot of advantages (speed, lack of vendor lock in, privacy) from a file system perspective over S3 (which is really a file object DB). If you must use S3 you might consider something like:

https://code.google.com/p/s3fuse/

as that will allow your front ends to use a common bucket for file storage.

The file system is by far the hardest part of distributing anything, and finding a way to abstract that away from the developer so it doesn't matter is even harder - every other part of the system I have working so this is really the last piece. We have a Jamroom Cloud profile that is private at this time, but I'll open it up to beta as soon as I can.

Hope this helps!

Genosis Events' Translation

Genosis


Yeah I know why this is happening. I just pushed out version 1.4.4 that rolls back the select field changes. The lang strings are still there, but in this version are being ignored. I'm going to have to implement this differently than I thought I would, otherwise the existing events and new events will not link up. I'm out of town right now and am out of time here for today, but update to version 1.4.4 so you can still work on your site - I will get a fixed version of this out this week that works for you.

Hope this helps!
updated by @brian: 12/29/14 09:23:55AM