Automatically to block ip
- Based on log files search pattern based block ip
haproxy log file

Automatically to block ip

Based on log files search pattern based block ip

#/usr/bin/sh
iptables --flush
/etc/init.d/iptables save
san=/var/log
cat /var/log/haproxy.log* | fgrep -e "^\/common\/fckeditor\/editor\/$" -e "phpMyAdmin\/scripts\/setup.php$" -e "[mM][yY][aA][dD][m<][iI][Nn]/scripts\/setup.php" -e "/\w00tw00t.at.blackhats.romanian.anti-sec*" -e "^/\wp-login.php $" -e "^/\administrator/fckeditor/\editor/filemanager/browser/default/connectors/test.html$"  -e "^/\pma/scripts/setup.php $" -e "^/\phpmyadmin/scripts/setup.php$" -e "^/\phph/php/\ph.php $" -e "^/\admin/index.php$" -e "^/\knkn/knk/kn.php $" -e "\/\/[pP][hH][pP][Mm][yY][aA][dD][mM][iI][Nn]/scripts/setup.php" -e "\/[pP][hH][pP][Mm][yY][aA][dD][mM][iI][Nn]/scripts/setup.php" -e '[mM][u][i][e][b][l][a][c][k]cat' -e '\/[sS][iI][gG][nN][uU][pP].php' -e "\/[wW][oO][rR][Dd][pP][rR][eE][sS][sS]/wp-login.php" -e "\/[wW][pP]-[lL][oO][gG][iI][Nn].php" -e "\/[pP][mM][aA]/scripts/setup.php" -e "\/[sS][yY][sS][tT][eE][mM]/fckeditor/editor" -e "[fF][cC][kK][eE][dD][iI][tT][oO][rR]" -e "[pP][hH][mM][yy][aA][dD][mM][iI][nN]" -e "phpmyadmin" -e "\/[cC][gG][iI]-[bB][iI][nN]/"  -e "\/[cC][gG][iI][bB][iI][nN]/" -e "?[gG][cC][lL][iI][dD]" -e "[Ii][nN][Dd][eE][xX].[pP][hH][pP]" -e "[sS][eE][tT][uU][pP].[pP][hH][pP]" -e "\/xampp\/*.*"  -ie "\/[Ee][Zz]2000"  -e "\/[aA][dD][Mm][Ii][Nn]/fckeditor" -e "manager\/html"  -e "[Aa-Zz].php" -e "mousemove.jsp" -e "\/docs/funcspecs/3.jsp" -e "\/mwheelIntent.jsp$" -e "\/[wW][0oO][0oO][tT][wW][0oO][0oO][tT].at.ISC.SANS*" -e "\/w00tw00t.at.ISC.SANS*" -e "\/muieblackcat" -e "\/admin\/phpmyadmin\/scripts\/setup.php" -e "[cC][gG][iI]" -e "\/[tT][mM][Uu]nblock.cgi" -e "\/styles/wp-login.php" -e "\/styles/administrator/index.php"  -e "\/yebryizpldede.html" -e "\/fckeditor" -e "\/cgi-mod*" -e "\/pma/*" -e "\/sql/phpmyadmin/*" -e "\/phppath*" -e "\/cgi-sys/" -e "\/wp-content" -e "\/yebryizpldede.html" -e "\/index.php/admin" -e "\/tmUnblock.cgi$" -e "^\/rom0" -e "^\/rom-0" -e "^\/plpl/plp/pl.php$" -e "^\/dana-na/auth/url_default/welcome.cgi" -e "\/caca2.txt$" -e "^\/javascripts\/.*[.]jsp$" -e ".*[.]php$" -e "^\/#E8#81#BD*" -e "^\/ejpxopsgc.html$" -e "^\/Diagnostics.asp" -e "^\/Ringing.at.your.dorbell!"| awk '{print $6}'|   awk -F':' '{print $1}'|sort -u >> $san/ip.blacklist 
cat $san/ip.blacklist | sort -u >> $san/ip.blacklist-updated
echo $san/ip.blacklist-updated
BLOCKDB="$san/ip.blacklist-updated"
IPS=$(grep -Ev "^#" $BLOCKDB)
for i in $IPS
do
iptables -A INPUT -d $i -j DROP
done
/etc/init.d/iptables save
> $san/ip.blacklist-updated
> $san/exip
> $san/tmpip

haproxy log file

  Mar 10 03:00:29 localhost haproxy[21225]: 10.225.254.53:38847 [10/  Mar/2015:03:00:29.228] http-ingress learnexa/prodapp01 0/0/0/37/339 302 614 - - --VN 1/1/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar 11 08:20:23 localhost haproxy[1786]: 72.18.249.61:16994 [11/  Mar/2015:08:20:23.385] http-ingress learnexa/prodapp01 4/0/1/409/427 302 1077 - - --VN 0/0/0/0/0 0/0 "GET /myadmin.php HTTP/1.1"
Mar  6 09:14:47 localhost haproxy[9283]: 10.163.255.122:42292 [06/  Mar/2015:09:14:41.983] http-ingress learnexa/prodapp02 0/0/0/18/5950 302 1066 - - --VN 5/5/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 09:33:49 localhost haproxy[9378]: 10.163.255.122:42412 [06/  Mar/2015:09:33:48.508] http-ingress learnexa/prodapp01 0/0/0/297/616 302 614 - - --VN 0/0/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 09:36:46 localhost haproxy[9378]: 10.163.255.122:42426 [06/  Mar/2015:09:36:46.504] http-ingress learnexa/prodapp01 0/0/0/18/324 302 614 - - --VN 0/0/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 09:36:58 localhost haproxy[9378]: 10.163.255.122:42431 [06/  Mar/2015:09:36:47.668] http-ingress learnexa/prodapp01 10903/0/0/17/11220 302 614 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:01:11 localhost haproxy[9378]: 10.163.255.122:42902 [06/  Mar/2015:10:01:10.806] http-ingress learnexa/prodapp01 0/0/0/22/350 302 614 - - --VN 0/0/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:01:17 localhost haproxy[9378]: 10.163.255.122:42907 [06/  Mar/2015:10:01:12.007] http-ingress learnexa/prodapp01 5298/0/1/17/5624 302 614 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:08:04 localhost haproxy[9378]: 10.163.255.122:43310 [06/  Mar/2015:10:08:04.114] http-ingress learnexa/prodapp01 0/0/0/18/600 302 614 - - --VN 1/1/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:08:13 localhost haproxy[9378]: 10.163.255.122:43314 [06/  Mar/2015:10:08:05.304] http-ingress learnexa/prodapp01 7410/0/2/18/7911 302 614 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:09:18 localhost haproxy[9378]: 10.163.255.122:43337 [06/  Mar/2015:10:09:17.715] http-ingress learnexa/prodapp01 0/0/0/18/312 302 614 - - --VN 0/0/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:09:22 localhost haproxy[9378]: 10.163.255.122:43342 [06/  Mar/2015:10:09:18.903] http-ingress learnexa/prodapp01 3099/0/0/22/3413 302 614 - - --VN 5/5/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:09:54 localhost haproxy[9378]: 115.111.237.103:13608 [06/  Mar/2015:10:09:53.416] http-ingress learnexa/prodapp01 0/0/2/498/951 302 685 - - --NI 7/7/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:10:05 localhost haproxy[9378]: 115.111.237.103:18117 [06/  Mar/2015:10:10:00.632] http-ingress learnexa/prodapp01 4283/0/0/17/4561 302 617 - - --VN 4/4/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:10:16 localhost haproxy[9378]: 115.111.237.103:17955 [06/  Mar/2015:10:10:15.950] http-ingress learnexa/prodapp01 0/0/0/17/506 302 617 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:25:38 localhost haproxy[9378]: 115.111.237.103:26355 [06/  Mar/2015:10:25:33.382] http-ingress learnexa/prodapp01 4450/0/0/39/4753 302 617 - - --VN 1/1/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:25:46 localhost haproxy[9378]: 115.111.237.103:29963 [06/  Mar/2015:10:25:45.834] http-ingress learnexa/prodapp01 0/0/0/22/291 302 617 - - --VN 3/3/1/1/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:28:27 localhost haproxy[9378]: 115.111.237.103:18022 [06/  Mar/2015:10:28:27.455] http-ingress learnexa/prodapp01 0/0/1/22/275 302 617 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:42:05 localhost haproxy[9378]: 115.111.237.103:14643 [06/  Mar/2015:10:42:05.421] http-ingress learnexa/prodapp01 0/0/1/18/294 302 617 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:42:13 localhost haproxy[9378]: 115.111.237.103:9046 [06/  Mar/2015:10:42:10.574] http-ingress learnexa/prodapp01 2932/0/1/19/3207 302 617 - - --VN 4/4/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:42:58 localhost haproxy[9378]: 115.111.237.103:29014 [06/  Mar/2015:10:42:58.369] http-ingress learnexa/prodapp01 0/0/1/20/277 302 617 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:43:04 localhost haproxy[9378]: 115.111.237.103:16551 [06/  Mar/2015:10:43:00.046] http-ingress learnexa/prodapp01 3687/0/0/19/3965 302 617 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:52:28 localhost haproxy[9378]: 115.111.237.103:7610 [06/  Mar/2015:10:52:25.812] http-ingress learnexa/prodapp01 2794/0/1/18/3071 302 617 - - --VN 0/0/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:52:34 localhost haproxy[9378]: 115.111.237.103:26143 [06/  Mar/2015:10:52:30.854] http-ingress learnexa/prodapp01 3384/0/1/18/3662 302 617 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:59:11 localhost haproxy[9378]: 115.111.237.103:2639 [06/  Mar/2015:10:59:11.664] http-ingress learnexa/prodapp01 0/0/0/18/272 302 617 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:59:21 localhost haproxy[9378]: 115.111.237.103:15471 [06/  Mar/2015:10:59:17.363] http-ingress learnexa/prodapp01 3882/0/1/18/4154 302 617 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 10:59:57 localhost haproxy[9378]: 115.111.237.103:14207 [06/  Mar/2015:10:59:57.295] http-ingress learnexa/prodapp01 0/0/0/17/287 302 617 - - --VN 3/3/1/1/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:00:02 localhost haproxy[9378]: 115.111.237.103:10894 [06/  Mar/2015:10:59:58.949] http-ingress learnexa/prodapp01 3013/0/0/17/3298 302 617 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:00:24 localhost haproxy[9378]: 115.111.237.103:8192 [06/  Mar/2015:11:00:24.354] http-ingress learnexa/prodapp01 0/0/1/15/283 302 617 - - --VN 3/3/1/1/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:01:59 localhost haproxy[9378]: 115.111.237.103:14790 [06/  Mar/2015:11:01:59.326] http-ingress learnexa/prodapp01 0/0/1/17/296 302 617 - - --VN 3/3/1/1/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:02:03 localhost haproxy[9378]: 115.111.237.103:1456 [06/  Mar/2015:11:02:00.994] http-ingress learnexa/prodapp01 2486/0/1/20/2758 302 617 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:02:54 localhost haproxy[9378]: 115.111.237.103:20803 [06/  Mar/2015:11:02:50.140] http-ingress learnexa/prodapp01 4107/0/1/19/4386 302 617 - - --VN 4/4/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:03:08 localhost haproxy[9378]: 115.111.237.103:13552 [06/  Mar/2015:11:03:08.067] http-ingress learnexa/prodapp01 0/0/0/16/278 302 689 - - --VN 0/0/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:06:33 localhost haproxy[9378]: 115.111.237.103:28514 [06/  Mar/2015:11:06:31.231] http-ingress learnexa/prodapp01 2311/0/1/18/2592 302 689 - - --VN 1/1/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:10:17 localhost haproxy[9378]: 72.18.249.61:9522 [06/  Mar/2015:11:10:17.596] http-ingress learnexa/prodapp01 3/0/0/21/35 302 619 - - --VN 5/5/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:10:24 localhost haproxy[9378]: 72.18.249.61:21974 [06/  Mar/2015:11:10:17.603] http-ingress learnexa/prodapp01 6811/0/0/16/6838 302 619 - - --VN 5/5/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:19:18 localhost haproxy[9378]: 115.111.237.103:3762 [06/  Mar/2015:11:19:17.963] http-ingress learnexa/prodapp01 0/0/0/20/282 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:19:32 localhost haproxy[9378]: 115.111.237.103:16022 [06/  Mar/2015:11:19:32.031] http-ingress learnexa/prodapp01 0/0/0/19/277 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:21:45 localhost haproxy[9378]: 115.111.237.103:19082 [06/  Mar/2015:11:21:45.609] http-ingress learnexa/prodapp01 0/0/1/16/339 302 689 - - --VN 3/3/1/1/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  6 11:21:51 localhost haproxy[9378]: 115.111.237.103:15204 [06/  Mar/2015:11:21:47.406] http-ingress learnexa/prodapp01 4251/0/1/16/4528 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 00:32:22 localhost haproxy[9378]: 72.18.249.61:22535 [07/  Mar/2015:00:32:22.582] http-ingress learnexa/prodapp01 5/0/0/19/37 302 619 - - --VN 4/4/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 00:32:35 localhost haproxy[9378]: 72.18.249.61:10099 [07/  Mar/2015:00:32:35.467] http-ingress learnexa/prodapp01 0/0/1/17/29 302 619 - - --VN 4/4/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:05:13 localhost haproxy[9378]: 115.111.237.103:26700 [07/  Mar/2015:02:05:13.654] http-ingress learnexa/prodapp01 0/0/1/16/279 302 689 - - --VN 1/1/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:05:20 localhost haproxy[9378]: 115.111.237.103:8534 [07/  Mar/2015:02:05:19.867] http-ingress learnexa/prodapp01 0/0/1/16/276 302 689 - - --VN 1/1/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:07:28 localhost haproxy[9378]: 115.111.237.103:8621 [07/  Mar/2015:02:07:28.311] http-ingress learnexa/prodapp01 0/0/0/20/276 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:12:09 localhost haproxy[9378]: 115.111.237.103:21684 [07/  Mar/2015:02:12:09.112] http-ingress learnexa/prodapp01 0/0/0/19/305 302 689 - - --VN 3/3/1/1/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:35:54 localhost haproxy[9378]: 115.111.237.103:6926 [07/  Mar/2015:02:35:53.947] http-ingress learnexa/prodapp01 0/0/1/17/283 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:35:58 localhost haproxy[9378]: 115.111.237.103:29037 [07/  Mar/2015:02:35:55.639] http-ingress learnexa/prodapp01 3028/0/0/18/3301 302 689 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:37:02 localhost haproxy[9378]: 115.111.237.103:20820 [07/  Mar/2015:02:37:02.108] http-ingress learnexa/prodapp01 0/0/0/19/281 302 689 - - --VN 2/2/1/1/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 02:37:07 localhost haproxy[9378]: 115.111.237.103:19004 [07/  Mar/2015:02:37:03.823] http-ingress learnexa/prodapp01 2952/0/0/17/3242 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:02:17 localhost haproxy[9378]: 115.111.237.103:4924 [07/  Mar/2015:03:02:17.097] http-ingress learnexa/prodapp01 0/0/0/21/284 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:02:21 localhost haproxy[9378]: 115.111.237.103:27716 [07/  Mar/2015:03:02:18.820] http-ingress learnexa/prodapp01 2766/0/0/16/3038 302 689 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:02:27 localhost haproxy[9378]: 115.111.237.103:6028 [07/  Mar/2015:03:02:23.305] http-ingress learnexa/prodapp01 3453/0/0/17/3722 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:51:45 localhost haproxy[9378]: 115.111.237.103:19371 [07/  Mar/2015:03:51:44.872] http-ingress learnexa/prodapp01 0/0/1/17/275 302 689 - - --VN 1/1/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:51:49 localhost haproxy[9378]: 115.111.237.103:6042 [07/  Mar/2015:03:51:45.972] http-ingress learnexa/prodapp01 3684/0/0/16/3965 302 689 - - --VN 1/1/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:53:08 localhost haproxy[9378]: 115.111.237.103:25929 [07/  Mar/2015:03:53:08.052] http-ingress learnexa/prodapp01 0/0/1/16/273 302 689 - - --VN 2/2/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:53:12 localhost haproxy[9378]: 115.111.237.103:25745 [07/  Mar/2015:03:53:09.745] http-ingress learnexa/prodapp01 2226/0/1/18/2502 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:53:16 localhost haproxy[9378]: 115.111.237.103:17084 [07/  Mar/2015:03:53:13.680] http-ingress learnexa/prodapp01 2312/0/0/18/2582 302 689 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  7 03:53:20 localhost haproxy[9378]: 115.111.237.103:23857 [07/  Mar/2015:03:53:17.716] http-ingress learnexa/prodapp01 2766/0/1/16/3035 302 689 - - --VN 3/3/0/0/0 0/0 "GET /.*%5D:%3CHOST%3E.*GET%5C/myadmin/scripts/setup.php HTTP/1.1"
Mar  7 05:35:16 localhost haproxy[9378]: 115.111.237.103:13093 [07/  Mar/2015:05:35:15.779] http-ingress learnexa/prodapp01 0/0/0/17/276 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 05:35:24 localhost haproxy[9378]: 115.111.237.103:28864 [07/  Mar/2015:05:35:23.932] http-ingress learnexa/prodapp01 0/0/1/16/273 302 689 - - --VN 0/0/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 05:36:44 localhost haproxy[9378]: 115.111.237.103:28955 [07/  Mar/2015:05:36:44.333] http-ingress learnexa/prodapp01 0/0/0/18/302 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 05:36:50 localhost haproxy[9378]: 115.111.237.103:3879 [07/  Mar/2015:05:36:46.094] http-ingress learnexa/prodapp01 3894/0/0/20/4191 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 05:46:28 localhost haproxy[9378]: 115.111.237.103:11433 [07/  Mar/2015:05:46:27.669] http-ingress learnexa/prodapp01 0/0/1/240/502 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 05:46:35 localhost haproxy[9378]: 115.111.237.103:30029 [07/  Mar/2015:05:46:30.311] http-ingress learnexa/prodapp01 4801/0/1/348/5450 302 689 - - --VN 0/0/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 06:00:40 localhost haproxy[9378]: 115.111.237.103:9363 [07/  Mar/2015:06:00:40.194] http-ingress learnexa/prodapp01 0/0/0/17/283 302 689 - - --VN 2/2/1/1/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 06:00:45 localhost haproxy[9378]: 115.111.237.103:16735 [07/  Mar/2015:06:00:41.648] http-ingress learnexa/prodapp01 3779/0/0/17/4058 302 689 - - --VN 5/5/1/1/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 06:05:57 localhost haproxy[9378]: 115.111.237.103:30055 [07/  Mar/2015:06:05:57.202] http-ingress learnexa/prodapp01 0/0/1/18/276 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 06:06:04 localhost haproxy[9378]: 115.111.237.103:21999 [07/  Mar/2015:06:06:04.549] http-ingress learnexa/prodapp01 0/0/1/18/281 302 689 - - --VN 2/2/1/1/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 07:03:29 localhost haproxy[9378]: 115.111.237.103:10318 [07/  Mar/2015:07:03:29.340] http-ingress learnexa/prodapp01 0/0/0/17/275 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 07:03:35 localhost haproxy[9378]: 115.111.237.103:15625 [07/  Mar/2015:07:03:31.044] http-ingress learnexa/prodapp01 4561/0/0/17/4833 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 07:12:40 localhost haproxy[9378]: 115.111.237.103:30735 [07/  Mar/2015:07:12:40.116] http-ingress learnexa/prodapp01 0/0/1/17/281 302 689 - - --VN 1/1/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 07:12:54 localhost haproxy[9378]: 115.111.237.103:23302 [07/  Mar/2015:07:12:49.677] http-ingress learnexa/prodapp01 4956/0/1/16/5235 302 689 - - --VN 1/1/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 07:26:11 localhost haproxy[9378]: 115.111.237.103:13740 [07/  Mar/2015:07:26:11.617] http-ingress learnexa/prodapp01 0/0/0/18/281 302 689 - - --VN 2/2/1/1/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  7 07:26:16 localhost haproxy[9378]: 115.111.237.103:7594 [07/  Mar/2015:07:26:13.316] http-ingress learnexa/prodapp01 3046/0/1/16/3317 302 689 - - --VN 3/3/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  9 02:12:59 localhost haproxy[9378]: 115.111.237.103:26518 [09/  Mar/2015:02:12:55.901] http-ingress learnexa/prodapp01 2999/0/0/19/3291 302 689 - - --VN 4/4/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"
Mar  9 02:13:06 localhost haproxy[9378]: 115.111.237.103:31767 [09/  Mar/2015:02:13:06.157] http-ingress learnexa/prodapp01 0/0/1/24/291 302 689 - - --VN 2/2/0/0/0 0/0 "GET /myadmin/scripts/setup.php HTTP/1.1"

Search patten you can see below

F “\/common\/fckeditor\/editor\/”

steps to do

/etc/init.d/iptables start

save above mentioned script following path /var/log/block-ip.sh

crontab -e 

*/1 * * * * /var/log/block-ip.sh 

/etc/init.d/crond restart


to view blocked ips below command
iptables -L

Improving iptables performance

===== Background  =====
 It was found that above steps of using iptables alone for blocking IP's consumed less amount of resource utilization (CPU & Memory) on our Load Balancer.

===== Solution  =====
IPSET - IP sets are a framework inside the Linux 2.4.x and 2.6.x kernel, which ensures lightning speed when matching an entry against a set.
	
   -> 1. Store multiple IP addresses or port numbers and match against the collection by iptables at one swoop;
   -> 2. Dynamically update iptables rules against IP addresses or ports without performance penalty;
   -> 3. Express complex IP address and ports based rulesets with one single iptables rule and benefit from the speed of IP sets

Steps Involved

 STEP : 1
    
          Install the ipset module to the server

          	yum install ipset
	
 STEP : 2
          
          Configure the IPSET Match statement
          
              ipset create blocklist host:ip
			
 STEP : 3
          
          Configure IPTABLES to match the blocklist
	
              iptables -A INPUT -m set --match-set blocklist src -j DROP
			
 STEP : 4
    
          Add IP's to the IPSET
	
              ipset add blocklist <ip>

Initial Files Needed

/var/log/haproxy.log*                - Load Balancer logs
/opt/ipscript/block_ip.sh            - The primary script
/opt/ipscript/ip.blacklist           - set of ip addresses to be blocked
/opt/ipscript/rules.new              - URL's to be blocked 
/opt/ipscript/release_ip.sh          - Temporarily releases the ip

IP Script written for automating the updates

  ## block_ip.sh - Updated by Syed
  ## This script can be run every minute safely (root user crontab) and is designed to be easy on server resources (CPU and Memory)
  ## May need further optimization

 #/bin/bash
 IPSCRIPT=/opt/ipscript
 DATE=$(date +"%d-%m-%Y")
 LOG=/var/log/
 #tail -n 500 /var/log/haproxy.* >> $IPSCRIPT/templog
 #TEMP_LOG=$IPSCRIPT/templog
 sort -u rules.new >> rules.temp;
 rm -f rules.new;
 mv -f rules.temp rules.new;
 echo -e "\nReading logs, Performing de-dup and Backup. PLEASE WAIT!\n"
 while true;do echo -n .;sleep 1;done &
 grep -f rules.new $LOG/haproxy.*|awk '{print $6}'|cut -d':' -f1|sort -u >> $IPSCRIPT/ip.blacklist;
 #rm -f TEMP_LOG;
 sort -u $IPSCRIPT/ip.blacklist >> $IPSCRIPT/ip.blacklist.bckup.$DATE;
 sort -u $IPSCRIPT/ip.blacklist >> $IPSCRIPT/ip.temp;
 > $IPSCRIPT/ip.blacklist; 
 cp -f $IPSCRIPT/ip.temp $IPSCRIPT/ip.blacklist;
 rm -f $IPSCRIPT/ip.temp;
 sort -u $IPSCRIPT/ip.blacklist.bckup.$DATE >> $IPSCRIPT/ip.temp
 > $IPSCRIPT/ip.blacklist.bckup.$DATE;
 cp -f $IPSCRIPT/ip.temp $IPSCRIPT/ip.blacklist.bckup.$DATE;
 rm -f $IPSCRIPT/ip.temp
 kill $!; trap 'kill $!' SIGTERM
 echo -e "\nDone!";
 echo -e "\nFlushing the Entries\n"
 iptables -L INPUT --line-numbers|grep -i black|awk '{print $1}' >> /$IPSCRIPT/rulenum
 for i in `cat $IPSCRIPT/rulenum`; do iptables -D INPUT $i; done
 sleep 1;
 rm -f $IPSCRIPT/rulenum
 ipset flush blacklist;
 ipset destroy blacklist;
 echo -e "\nDone!"
 echo -e "\nAdding Entries\n"
 ipset create blacklist hash:ip;
 iptables -A INPUT -m set --match-set blacklist src -j DROP;
 while true;do echo -n .;sleep 1;done &
 for i in `cat $IPSCRIPT/ip.blacklist`;
 do
 flock -w 5 /var/lock/iptables -c "ipset add blacklist $i";
 done;
 sleep 1
 kill $!; trap 'kill $!' SIGTERM
 echo -e "\nDone!\n";

Script to take backup daily

		## This script can run everyday safely (/etc/cron.daily/)
			#/bin/bash
			DATE=$(date +"%m-%d-%Y");
			#save configuration
			ipset save blacklist > /opt/ipscript/ipsetbckup/ipset.conf.$DATE;

Script to Remove IP - Ask permission before you do it

## release_ip.sh
## You have to remove the entry in the logfile manually first else its useless
			
#/bin/bash
DATE=$(date +"%d-%m-%Y")
STORE=/opt/ipscript
sed 's/$1//' $STORE/ip.blacklist|sort -u >> $STORE/ip.temp1;
rm -f $STORE/ip.blacklist;
mv -f $STORE/ip.temp1 $STORE/ip.blacklist;
mv -f $STORE/ip.temp1 $STORE/ip.blacklist.bckup.$DATE;
rm -f $STORE/ip.temp1;
ipset del blacklist $1;

rules.new

## Rules to block unwanted URL's - Last updated 05-MAY-2015 - WARNING - Leave no space at the start of the line
			
^\/[aA][dD][Mm][Ii][Nn]/fckeditor
[Aa-Zz].php$
^\/admin/index.php$
^\/administrator/fckeditor/\editor/filemanager/browser/default/connectors/test.html$
^\/admin\/phpmyadmin\/scripts\/setup.php
^\/[cC][gG][iI]
^\/[cC][gG][iI]-[bB][iI][nN]/
^\/[cC][gG][iI][bB][iI][nN]/
^\/cgi-mod/
^\/cgi-sys/
^\/common\/fckeditor\/editor\/
 \/docs/funcspecs/3.jsp
 \/[Ee][Zz]2000
 fckeditor
 [fF][cC][kK][eE][dD][iI][tT][oO][rR]
 \/[aA][dD][Mm][Ii][Nn]/fckeditor
 ^?[gG][cC][lL][iI][dD]
 [Ii][nN][Dd][eE][xX].[pP][hH][pP]
 \/index.php/admin
 ^\/knkn/knk/kn.php $
 ^\/manager\/html
 [mM][u][i][e][b][l][a][c][k]cat
 [mM][yY][aA][dD][m<][iI][Nn]/scripts\/setup.php
 mousemove.jsp
 \/muieblackcat
 mwheelIntent.jsp
 ^\/phph/php/\ph.php
 phpmyadmin
 ^\/phpmyadmin/scripts/setup.php$
 ^\/phpMyAdmin\/scripts\/setup.php$
  \/phppath*
  \/pma/*
 ^\/pma/scripts/setup.php
 [pP][hH][mM][yy][aA][dD][mM][iI][nN]
 \/[pP][hH][pP][Mm][yY][aA][dD][mM][iI][Nn]/scripts/setup.php
 \/\/[pP][hH][pP][Mm][yY][aA][dD][mM][iI][Nn]/scripts/setup.php
 \/[pP][mM][aA]/scripts/setup.php
 \/sql/phpmyadmin/*
 [sS][eE][tT][uU][pP].[pP][hH][pP]
 \/[sS][iI][gG][nN][uU][pP].php
 \/[sS][yY][sS][tT][eE][mM]/fckeditor/editor
 ^\/styles/administrator/index.php
 ^\/styles/wp-login.php
 \/[tT][mM][Uu]nblock.cgi
 ^w00tw00t.at.blackhats.romanian.anti-sec*
 ^w00tw00t.at.ISC.SANS*
 ^\/wp-content
 ^\/wp-login.php $
 \/[wW][0oO][0oO][tT][wW][0oO][0oO][tT].at.ISC.SANS*
 \/[wW][oO][rR][Dd][pP][rR][eE][sS][sS]/wp-login.php
 \/[wW][pP]-[lL][oO][gG][iI][Nn].php
 \/xampp\/*.*
 \/yebryizpldede.html
 \/tmUnblock.cgi$
 ^\/rom0
 ^\/rom-0
 ^\/plpl/plp/pl.php$
 ^\/ejpxopsgc.html$
 ^\/#E8#81#BD*
 .*[.]php$
 .*[.]jsp$
 ^\/dana-na/auth/url_default/welcome.cgi
 \/caca2.txt$
 ^\/Diagnostics.asp
 ^\/Ringing.at.your.dorbell!
 manager\/html

ip.blacklist

## block unwanted IP's - Last updated 30-APR-2015
101.222.232.22
103.15.88.76
103.242.101.145
103.253.113.173
103.253.42.28
103.27.127.238
104.255.68.139
109.200.24.36
110.77.142.109
110.77.159.244
110.77.179.232
110.77.197.247
110.77.240.224
110.89.13.167
111.11.181.245
113.135.195.55
113.161.84.88
113.174.198.171
113.53.252.109
114.215.150.17
114.45.135.248
116.226.38.156
118.26.202.100
119.57.17.20
119.82.224.162
120.195.36.188
121.40.187.101
122.155.197.6
123.30.132.164
123.85.194.121
124.11.134.238
125.227.108.82
125.227.114.55
125.81.189.218
133.18.96.54
140.120.90.187
140.224.94.127
1.53.76.193
157.7.209.93
186.128.34.221
186.9.94.199
187.149.44.172
187.149.70.200
187.151.126.62
187.189.10.175
188.132.193.155
188.132.204.208
188.138.104.206
189.148.172.221
189.152.130.84
189.174.22.197
189.225.134.89
190.157.187.56
192.187.110.98
194.28.6.131
194.6.233.15
199.101.185.133
199.217.115.178
200.109.205.109
200.74.138.189
201.214.57.118
201.246.231.226
201.99.63.27
202.129.59.146
202.46.14.130
203.113.122.174
203.127.84.36
203.146.208.153
203.157.45.75
210.61.165.62
211.137.179.236
211.154.6.139
217.12.202.39
218.241.190.177
218.60.29.198
219.144.200.102
219.232.247.108
220.255.1.140
220.255.1.46
220.255.1.49
220.255.1.73
222.137.23.8
222.186.190.194
222.218.142.137
223.205.74.225
31.185.6.10
37.135.168.43
37.55.237.47
42.119.55.3
46.105.247.175
46.118.117.18
46.17.100.114
46.98.6.107
5.35.253.135
58.153.232.85
58.222.21.43
58.234.140.150
60.249.202.132
69.197.186.210
77.38.132.77
78.135.88.147
79.117.15.33
80.230.102.97
82.146.32.60
84.77.89.185
85.114.142.136
88.147.181.141
88.238.79.197
89.111.180.158
91.194.84.124
91.200.160.148
91.217.90.49
91.235.176.40
92.114.220.166
93.76.61.237
94.23.165.144

Performance comparison iptables vs iptables + ipset

iptables reload takes more than 2 second for 956 IP's -> Takes more hardware resources - Fast
iptables + ipset takes few seconds to reload 121 IP's  -> Takes less hardware resources - Fast

Tested on 10.229.94.14.

With IPSET parsing parallel log file size greater than 10Mb


[root@10-229-94-14 ipscript]# date; sh block_ip.sh; date;
Thu Apr 30 06:32:25 EDT 2015
Thu Apr 30 06:32:51 EDT 2015

[root@10-229-94-14 ipscript]# ipset list |wc -l
122

For 122 IP's it takes 22 - 30 seconds

Main Advantage is the response time for incoming request, compared to netfilter default drop list;

REF : http://daemonkeeper.net/781/mass-blocking-ip-addresses-with-ipset/

Ip tables fine tuned performance

[root@10-166-152-16 ~]# cat /etc/sysconfig/iptables | grep DROP|wc -l
956

[root@10-166-152-16 ~]# date;/var/log/block-ip.sh;date
Tue May 12 08:23:21 EDT 2015
Saving firewall rules to /etc/sysconfig/iptables:          [  OK  ]
/var/log/ip.blacklist-updated
Saving firewall rules to /etc/sysconfig/iptables:          [  OK  ]
Tue May 12 08:23:22 EDT 2015

For 956 ip's it take only 1 seconds by iptables
Script slightly modified .Search entire log file & get blocked banned search patterns

Previously have used grep . Now we are using fgrep

 I have created haproxy.log.1 12Mb file size to test

 Now script took only 4seconds to execute . Search entire log file & get blocked banned search patterns

 [root@10-166-152-16 ~]# du -sh /var/log/haproxy.log*
 192K	/var/log/haproxy.log
 12M	/var/log/haproxy.log.1

 [root@10-166-152-16 ~]# date;/var/log/block-ip.sh;date
 Tue May 12 05:18:36 EDT 2015
 Saving firewall rules to /etc/sysconfig/iptables: [ OK ]
 /var/log/ip.blacklist-updated
 Saving firewall rules to /etc/sysconfig/iptables: [ OK ]
 Tue May 12 05:18:40 EDT 2015

Table of Contents

Automatically to block ip

Based on log files search pattern based block ip

haproxy log file

Search patten you can see below

steps to do

Improving iptables performance

Steps Involved

Initial Files Needed

IP Script written for automating the updates

Script to take backup daily

Script to Remove IP - Ask permission before you do it

rules.new

ip.blacklist

Performance comparison iptables vs iptables + ipset

Ip tables fine tuned performance