Re: exploit or human

[Tim <tim-forensics () sentinelchicken org>]

> We've got a hard disk failure (bad blocks - reported the array controller
> bios) on a scsi hard-disk on an INTEL platform (running Fedora Core 2 Linux
> operating system). What is interesting is that this hard-disk failure
> occurred after a "I don't know what it is... let's reboot it and see after
> that" situation. Situation describe by many "segmentation fault" when using
> typical application like vi or service or even grub-install. Grub did not
> start again after that (we tried to reinstall it with an Install CD 1 from
> Fedora and grub-install did said "segmentation fault" again)
>
> We did recover the data on that scsi hard-drive by mounting it on another
> machine. 
>
> So far so good (sort of)

After reading this part, there are two likely explanations, IMO:

 - Bad RAM

 - Bad sectors in swap

The best way to check for the first issue, is to use something like
memtest86.  I have found a number of bad sticks with it.


> After a week or so, another Linux server, began to show the same errors
> while giving shell commands and also sshd listened on port 22 we cannot do a
> ssh on it. We did not make the connection to the previous case (as we
> thought was a possible hardware failure), reboot it and grub did not start.
> We boot again with an install CD from redhat 7.3 (as we had redhat 7.3
> installed on that hard-disk, and thought if any files are missing...), the
> hard-disk was recognized by controller (again scsi hard-disk), fdisk view
> the partitions, and cannot this time mount them. (As I write this the "much
> more important data that hardware" hard-disk is at a computer service, for
> data recovery.
>
> Again, on a third Linux server (redhat 7.3) we got some messages at the
> primary console (kernel BUG commit.c #some number, lots of stack text and
> hexa symbols...) and again can't do ssh on it (it responds to ping and
> traceroute, telnet ip_address port 22 works...). We are kind of worried
> regarding the reboot of this machine...
>
> Could that be a worm, exploit or something, or looks like a human
> intervention situation?!

After these two, it isn't looking good to me.  Are these machines
running all the same hardware?  If so, maybe you just got a bad batch of
something.

If not, then that kind of highly-unstable kernel behavior would lead me
to start searching for signs of kernel exploits.  There's been quite
a few local kernel holes in the last 6 months, after all.

Of course, that's just pure speculation.

tim