nanog mailing list archives

Re: TCP torture testing


From: "Pete D." <denitto () gmail com>
Date: Fri, 17 Jan 2025 20:46:34 -0500

Shopify built a go app called Toxiproxy that would allow for injecting TCP
oddballs into an http stream.

https://github.com/Shopify/toxiproxy

Toxiproxy is a framework for simulating network conditions. It's made
specifically to work in testing, CI and development environments,
supporting deterministic tampering with connections, but with support for
randomized chaos and customization. *Toxiproxy is the tool you need to
prove with tests that your application doesn't have single points of
failure.* We've been successfully using it in all development and test
environments at Shopify since October, 2014. See our blog post
<https://shopify.engineering/building-and-testing-resilient-ruby-on-rails-applications>
on resiliency for more information.

--Pete





On Fri, Jan 17, 2025 at 2:42 PM Brandon Martin <lists.nanog () monmotha net>
wrote:

On 1/17/25 14:29, William Herrin wrote:
Well... In theory, TCP closes the segment at the end of the
application's send() and sets the PSH flag. Likewise, on the receiving
side the recv() returns before filling the buffer upon receipt of a
segment with the PSH flag set.

Every segment this thing sends has PSH set which again makes me think
that they've got TCP_NODELAY set but are sending their messages
piecemeal across multiple send/write calls.

The actual high-level messages are fairly small at typically less than
100B.  Most implementations end up sending the entire message in a
single TCP segment.

In theory. In practice, it doesn't always work out that way and
applications which depend on a short recv() meaning that was where the
sender's send() ended tend to flake out in unexpected ways.

I don't think that's the issue in this case, but it's a useful thing to
go looking for.

--
Brandon Martin



-- 
--
Peter A. DeNitto
denitto () gmail com <dentto () gmail com>

Current thread: