12

I use a perl script (with the Semaphore package) to fire off long-running ssh commands to AWS instances. For several reasons, I do NOT run the ssh commands in the background.

Recently, the Comcast fiber to our office building was cut by a construction crew. We maintain a backup CenturyLink connection, and our IT people switched our office connection over to CenturyLink.

My ssh processes died with a "timeout, server not responding" message when we switched from Comcast to CenturyLink. They died again when we switched back to Comcast after the fiber was repaired.

Is this expected behavior for an open ssh command if the local public IP address changes? If I put the ssh commands in the background, would it solve this issue?

Morgan Brown
  • 173
  • 2
  • 10

3 Answers3

20

Yes, that's expected behavior. Your connection was physically interrupted, after all.

If an interruption in your connection may cause a failure of a long running remote process, consider running it in a tmux session.

Michael Hampton
  • 252,907
14

If you do not want to run ssh commands in the background (manually) you may give mosh (mobile shell) a try. It's available for probably any distribution and system including various BSD, Android, Windows and macOS.

Beko Pharm
  • 306
  • 1
  • 6
3

Yes, it's normal.

A TCP server identifies a connection by the combination of local address, local port, remote address and remote port.

Since you talk about "the local public IP address changes" I presume your workplace uses some form of NAT to map the private addresses on your LAN to one or more public addresses on the current internet connection.

Depending on how exactly the NAT is implemented there are several possibilities for what exactly might happen at the packet level, but for regular TCP all of them end up with the TCP connection failing.

There is an extension called "multipath TCP" which (among other things) allows connections to be maintained across network changes, but afaict using it between linux boxes still requires third party kernel modules at this time.

Peter Green
  • 4,464