Bugtraq: Smashing the stack on a DEC Alpha

[lamontg () hitl washington edu (Lamont Granquist)]

I've been trying to impliment Aleph1's stack-smashing attack, but on a DEC
Alpha, and have run into a bit of a glitch here.  First, the Alpha appears
to stick the return address lower than the dynamic variables in the frame
-- okay, so you have to overwrite the return address of the previous
function such that it executes out of the environment.  Not a major
problem.  However, I'm trying to work my way up to executing code in the
dataspace and I'm running into a bit of a problem, e.g:

void main(void) {
  int *ad1;

  char shellcode[] = "\x01\x80\xfa\x6b";   /* ret */

__asm__("
        addq $15,24,$8
        stq  $8,16($15)
        ");

  printf("%x\n", &ad1);
  printf("%x\n", ad1);
  printf("%x\n", &shellcode);
  printf("%x\n", *ad1);

__asm__("
        addq $15,24,$21
        jsr $26,($21)
        ");

  printf("jsr worked\n");
}

That works correctly in that ad1 == &shellcode and *ad1 = 0x6bfa8001, but
it seems that it croaks when it attempts to execute the jsr into the
stack:

> ./testsc
1ffff6c0
1ffff6c8
1ffff6c8
6bfa8001
Segmentation fault (core dumped)
>
I haven't tried throwing the code into the environment and trying to
execute it there, but I can't see how that would make much of a
difference over the stack...

--
Lamont Granquist <lamontg () hitl washington edu> (206)616-1469 fax:(206)543-5380
Human Interface Technology Lab.  University of Washington.  Seattle, WA
PGP pubkey: finger lamontg () hitl washington edu