Re: one of my servers has been compromized

[John Jacobs <flamdugen () hotmail com>]

> For future reference, and for the benefit of people searching for
> solutions to similar problems: You've made the most common rookie
> mistake. You have already trashed potentially critical information
> about the attack by trying to clean up the server first. Don't do
> that.

Tim, while I do believe there is some truth in what you are saying here, I respectfully disagree in that this tends to 
be a run-of-the-mill IRC bot as evidenced by the Undernet advisory.  This looks like a skiddie-de-jour attack against 
PHPMyAdmin and nothing to be concerned with regarding cloning disk images and full forensics.  I do respect your input 
and thoughts though for a more targeted attack; not an IRC bot in /tmp.

That being said, I strongly believe in preserving bash_history as well as vital log data.  It's best/wise to ship this 
off to a separate Syslog server.  If you're paranoid turn up stunnel between the devices.  For example and as evidenced 
by many of the documented attacks here purging of bash_history is common ala 'history -c' after fun.  To thwart this I 
like the idea of logging to syslog often, ensure permissions are strict for the syslog messages, and shipping the 
syslog data off to a separate box.  I like to:

1) Generate an E-Mail alert when someone logs in, by adjusting /etc/bash.bashrc (or similar based on distribution) to:

#Email alert for login
echo -e "Subject: Login from $(/usr/bin/whoami) on $(/bin/hostname) at $(/bin/date)\n\n$(/usr/bin/last -ian 
10)\n"|/usr/sbin/sendmail recipient () example com

2) Preserve, via Syslog, commands executed at the prompt, by adjusting /etc/profile.  Adjust /etc/syslog.conf or 
/etc/rsyslog.conf to forward these syslog messages off-box to another asset.  If you're paranoid use stunnel.

export PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND ; }"'echo "$$ $USER $(history 1)"|/usr/bin/logger -p 
user.alert -t bash_history'
readonly PROMPT_COMMAND

3) Preserve bash_history by adjusting /etc/profile:

#Secure the Bash History
export HISTSIZE=1500
export HISTCONTROL=''
export HISTIGNORE=''
export HISTTIMEFORMAT='%F %T '
readonly HISTFILE
readonly HISTFILESIZE
readonly HISTSIZE
readonly HISTCONTROL
readonly HISTIGNORE
readonly HISTTIMEFORMAT

4) Optionally use chattr to set ~/.bash_history to append-only:

  #Secure .bash_history (poke fun of the while subshell if you wish)
  /usr/bin/find / -maxdepth 3|/bin/grep -i bash_history|while read line; do /usr/bin/chattr +a "$line"; done

5) Use of an IP Recorder, something like daemonlogger, in ring-buffer mode, as a way to record all ingress/egress 
traffic using a percentage of the disk.  See http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html

I am eager to hear any additional thoughts or methods for security information such as this.

Thanks,
John





                                          
_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/