[jcable]

still under fire. >3k IPs blocked thus far. they are not getting to the DB at all, brickwalled @ one request to Apache before being blocked at iptables. They are however still firing roughly one new IP per minute towards our server. One of our other sysadmins took the liberty to search for the spammer (google) to see if he might be bragging or if the exploit is posted publicly, it happens! nothing yet BUT he did run into several other JR sites under siege just by searching the, same accounts, emails. a few sites he said looks as if they are crushing DBs with rouge users and spam, eventually will crash sql i imagine if they picked up the pace. from sounds/looks of it someone or a hack team is purposefully seeking JR sites milking this exploit, its def a botnet (more then likely they've paid to for time to use) they obviously haven't checked in on it, as they keep hitting us consistently and for last 48hrs we have them brute forced at the firewall.

We know the script scans for content on the site. look for the same IPs from the accounts, massively scanning your site profiles/forum, something like 8 requests a second per IP! We think its using this to slurps up keywords and sentences based on what it finds on your site. It then uses that content to write (garbage) what it sees as somewhat relevant content with a lot SPAM mixed in, I guess to try to fly under the radar, search hits, etc. Once its got that and an account it goes right for the blogs. Never the forum, interestingly, but its flat txt and a little slower so that might be why.

All we can do is hold them but I hope that no one else is going to get hit this hard, its complete bs, for a side project for all of us, its annoying. real IT to do, lol

Thanks Again Guys, let me know if I can help in any way to defeat these bastards!
-jc
-g'luck

[emdiamond]

Just received this email. I blocked this address but looks like they may be probing now:

testaffpro@gmail.com



Subject: testing email validity
From: testaffpro@gmail.com
Date: Fri, November 18, 2011 7:43 am
To: xxxxxxxxxxxxxxx
Options: View Full Header | View Printable Version | Download this as a file | | View as HTML | Add to Address Book

Hi

test email

Please ignore

Thanks

[speedbunny]

Also still under fire: Now I've removed the signup.php from the home directory as a temporary measure, and redirecting all logins to Google. Obviously they couldn't sign up, but the spam bot was going through the motion of trying to sign up two or three times a minute... let's see if they get the message now!


_________________
http://rockaffairs.com <- My Jamroom site!
http://ownersabroad.org <- My holiday site!
http://vapers.co.uk <- My e-cig site!

Er, yeah, I'll stop that now, I have about 50 more.... (heads over to the soda machine...)

[emdiamond]

Has anyone tried the DNS Check? I'm getting allot of signups with bad emails, Does anyone know if DNS check works? If so, how do we set it up?

Quote:

E-Mail Settings ERROR! You have entered an invalid Return E-Mail Address - please enter a valid E-Mail Address E-Mail Address DNS Check Yes No E-Mail Address DNS URL Admin Contact E-Mails Yes No

[Bloodcrave]

Yes, use this url for email address DNS url

Code

http://www.intodns.com/

And set Email address DNS check to "yes"

emdiamond:

Has anyone tried the DNS Check? I'm getting allot of signups with bad emails, Does anyone know if DNS check works? If so, how do we set it up? Quote: E-Mail Settings ERROR! You have entered an invalid Return E-Mail Address - please enter a valid E-Mail Address E-Mail Address DNS Check Yes No E-Mail Address DNS URL Admin Contact E-Mails Yes No

[kdh]

I work with Jcable on our site..

We have banned almost 4500 IP address..

I will provide you the list of IP address so you can ban them before they start beating on your server.

Private message me and I will send you a link to the text file. I do not want to post it on the open forum.

[2tunes]

I've put my site in Maintenance Mode until I can get the fix. I've also tried to find anything common in the sites that aren't hotmila and they are all wordpress site or no content. the hosts most often found has been godaddy. tracert and nothing to tie to the spammer as they've been a wide range of hosts. can we set up a skype to coordinate our efforts?

Tom
skype = t440music


_________________
440MUSIC.COM Internet Radio & Music Store
The Granddaddy of Internet Radio for the
Unknown, Unsigned, and Independent Musician
Going Beyond The Reach Of Satellite

[kdh]

With our site, I noticed in my apache logs they were hitting a URL that contained "signup&quota_id=9".

That was the loop hole they were using..

I have written a small autoban hammer script that when an IP address hits my site with "signup&quota_id=9" in its url.

it'll get banned.

I can provide this script, however.. its a use at your own risk.. How they hit my site, vs yours might be a little differnt.. so you'd need to parse your apache logs to figure it out.

if you know some VI, and know how to read apache logs, I'll provide my script, but its a totally USE AT YOUR OWN RISK.. My own script banned jcable by accident when i first turn it on. lol.

PM me and i'll provide details.

[2tunes]

Quote:

I have written a small autoban hammer script that when an IP address hits my site with "signup&quota_id=9" in its url. it'll get banned.

This is why I think we need to coordinate our efforts!! We can get results much faster if we can work together. As much as I think it's up to the JR team to find the ultimate solution I think it's our responsibility to find a quick solution. My site is in maintenance mode so I don't have to deal with any more false signups, I had over 1100 in 18 hours.

I hope I can get to the Apache logs to check them.
Checking my logs I find they are hitting quota 1


_________________
440MUSIC.COM Internet Radio & Music Store
The Granddaddy of Internet Radio for the
Unknown, Unsigned, and Independent Musician
Going Beyond The Reach Of Satellite

[Brian]

2tunes:

Quote: I have written a small autoban hammer script that when an IP address hits my site with "signup&quota_id=9" in its url. it'll get banned. This is why I think we need to coordinate our efforts!! We can get results much faster if we can work together. As much as I think it's up to the JR team to find the ultimate solution I think it's our responsibility to find a quick solution. My site is in maintenance mode so I don't have to deal with any more false signups, I had over 1100 in 18 hours. I hope I can get to the Apache logs to check them. Checking my logs I find they are hitting quota 1

I'm open to feedback on what an "ultimate" solution would be. Beyond using ReCaptcha, I'm not aware of any system that can judge the intent of a user. How do you tell legitimate users from "real" spammers who are signing up with the intention to post blog posts full of spam?

Thanks!

- Brian


_________________
Make sure and check out:
* The Jamroom FAQ
* The Jamroom Documentation

[2tunes]

Quote:

I'm open to feedback on what an "ultimate" solution would be. Beyond using ReCaptcha, I'm not aware of any system that can judge the intent of a user. How do you tell legitimate users from "real" spammers who are signing up with the intention to post blog posts full of spam?

This is why I think working together will help us all find a satisfactory solution. I'm going to be off work for the next 2 weeks so I'm available not that I'll be that helpful but I do have additional resources I can tap in to.

Oh I've been looking at the Apache logs and see that the spammer is using at least 2 Windows NT servers v6.0 + v6.1 with FireFox v3.6.24 on NT 6.0 and v3.6.17 on the NT 6.1. Don't know if this helps.

Happy TDay All


_________________
440MUSIC.COM Internet Radio & Music Store
The Granddaddy of Internet Radio for the
Unknown, Unsigned, and Independent Musician
Going Beyond The Reach Of Satellite

[jcable]

I don't think that there is another 'ultimate solution' that JR can provide at this stage. They have already closed the loop hole as provided in Brian's 4.2.6 release of the bonus pack on signup.php. they have done all they can to provide support from an codebase perspective. The rest has to be from a systems perspective to hold them out till they give in unfortunately. unless someone else finds another bug, I think JR is patched.

If you can run the bash scripts, have ssh access.. i'm sure KDH will provide details but as mentioned be cautious. It won't be good if you lose ssh access to the box via iptables or ban real users by accident.

Banning the IPs we already have collected in IP tables can also at least detire them but they keep coming at us, never ending it seems, so its a moving target. no pending users or spam for us anymore, so its just a matter of time till they give up.


I would apply the following, then sit and wait for them to stop hammering. All you can do.
add: you can also look at my notes in the VIP forum, as it was happening I left notes to Brian there.
http://www.jamroom.net/phpBB2/viewtopic.php?t=40006

jcable:

What we've learned: - Banning the emails in JR is a completely pointless effort. - Manually banning the IPs is also pointless effort. - Banning IPs as suggested via JR can work but still allows apache access We have taken the following actions: - removed the affected/exploited quota and moved all active user to a new one. - deleted and removed all the rouge accounts, had to take it down for maintenance for a few minutes. - we wrote a bash script to loop and grep the apache logs and drop IPs @ iptables that were accessing the exploited now removed quota . - we patched the signup.php in the test release provided by Brian (Thanks!) - we switched captcha to Google API (Great Suggestion! no template edits were required) -jc

[Douglas]

You could always setup your signup quota as a dummy quota, no ranking etc. and make sure the pending features are all checked. Then when a user signs up, you can check them out before modifying them and moving them to a legit quota.

I've had a few users try to signup and post a spam blog, but it never made it to any lists on the site and I was able to remove their account and band their IP before they posted a bunch of spam.

Hope this helps,
Douglas


_________________
Douglas Hackney
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center

[kdh]

bigguy:

2tunes: Quote: I have written a small autoban hammer script that when an IP address hits my site with "signup&quota_id=9" in its url. it'll get banned. This is why I think we need to coordinate our efforts!! We can get results much faster if we can work together. As much as I think it's up to the JR team to find the ultimate solution I think it's our responsibility to find a quick solution. My site is in maintenance mode so I don't have to deal with any more false signups, I had over 1100 in 18 hours. I hope I can get to the Apache logs to check them. Checking my logs I find they are hitting quota 1 I'm open to feedback on what an "ultimate" solution would be. Beyond using ReCaptcha, I'm not aware of any system that can judge the intent of a user. How do you tell legitimate users from "real" spammers who are signing up with the intention to post blog posts full of spam? Thanks! - Brian

there is no ultimate solution because every attack can be different. jamroom can only do so much before the webmaster needs to dive into things, read logs, and find the pattern of things. btw, big ups and thanks to jamroom for patching that hole so quickly.

I took the cheap route and I'm just using iptables to stop the attack when the old loop hole pops up in my logs. However, i have a sneaking feeling I should move onto apaches Mod security because iptables is just going to chew up all my memory and pound on my cpu. the bottom line is its up to the webmaster

i've noticed the attack has slowed down in my logs at this point. really who ever is doing this rented some time on a botnet, clicked the go button on his script and walked away. he/she will review thier logs once things have died down.

[kdh]

2tunes:

Quote: I have written a small autoban hammer script that when an IP address hits my site with "signup&quota_id=9" in its url. it'll get banned. This is why I think we need to coordinate our efforts!! We can get results much faster if we can work together. As much as I think it's up to the JR team to find the ultimate solution I think it's our responsibility to find a quick solution. My site is in maintenance mode so I don't have to deal with any more false signups, I had over 1100 in 18 hours. I hope I can get to the Apache logs to check them. Checking my logs I find they are hitting quota 1

cause you found out its quota 1, when you find your apache logs.. here is my script that I am using..

use at your own risk.

you must be root to run this...

Code

#!/bin/bash tail -f /path/to/your/jamroom/apache/access_log | while read line ; do  echo "$line" | grep "signup&quota_id=1" >> /dev/null  if [[ $? = "0" ]] ; then   ip=`echo "$line" | awk '{print $1}'`   echo "$ip" >> ./list.txt   bancount=`wc -l ./list.txt | awk '{print $1}'`   iptables -I INPUT -s $ip -j DROP   echo "`date` Banned IP $ip total banned: $bancount "   echo "`date` Banned IP $ip total banned: $bancount " >> ./autoban.log   iptables-save > ./update.config  fi done

obviously you'll need shell and root access to your machine. clearly you'll need to change "/path/to/your/jamroom/apache/access_log" to the correct path to your logs.

If you run this script, and lock yourself out, you should be able to just reboot the box and you should get back in. if you don't understand this script.. stop right there, and don't use it.

the script will create this file ./autoban.log, and in that file will be all the ip address's you've banned and what time. output will look like so..

Code

Sat Nov 19 21:46:27 EST 2011 Banned IP 46.4.88.56 total banned: 4542 Sat Nov 19 21:53:25 EST 2011 Banned IP 31.214.155.59 total banned: 4543 Sat Nov 19 21:59:28 EST 2011 Banned IP 31.214.155.56 total banned: 4544 Sat Nov 19 22:04:27 EST 2011 Banned IP 68.35.93.184 total banned: 4545 Sat Nov 19 22:04:32 EST 2011 Banned IP 174.36.32.191 total banned: 4546 Sat Nov 19 22:05:32 EST 2011 Banned IP 89.30.138.109 total banned: 4547 Sat Nov 19 22:07:15 EST 2011 Banned IP 184.82.93.18 total banned: 4548 Sat Nov 19 22:20:51 EST 2011 Banned IP 124.120.100.133 total banned: 4549 Sat Nov 19 22:21:54 EST 2011 Banned IP 68.147.11.114 total banned: 4550 Sat Nov 19 22:24:22 EST 2011 Banned IP 173.208.197.41 total banned: 4551

i have family flying into town tomorrow so my response to this post will be limited.. best of luck everyone.