Jenkins Slave goes offline during the build

Question

A Jenkins 1.651 master and slave have been installed on a Windows Server 2012. The slave is dedicated to run Selenium tests, but most of the time it goes offline:

Build step 'Execute Windows batch command' marked build as failure
ERROR: slave-tester is offline; cannot locate jdk7

and

Apr 20, 2016 2:20:05 PM INFO jenkins.slaves.DefaultJnlpSlaveReceiver handle
Disconnecting slave-tester as we are reconnected from the current peer
Apr 20, 2016 2:20:17 PM WARNING org.jenkinsci.remoting.nio.NioChannelHub run
Communication problem
java.io.IOException: An existing connection was forcibly closed by the remote host
    at sun.nio.ch.SocketDispatcher.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(Unknown Source)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
    at sun.nio.ch.IOUtil.read(Unknown Source)
    at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
    at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:136)
    at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:306)
    at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

Apr 20, 2016 2:20:17 PM WARNING jenkins.slaves.JnlpSlaveAgentProtocol$Handler$1 onClosed
NioChannelHub keys=2 gen=88003: Computer.threadPoolForRemoting [#2] for slave-tester terminated
java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@17d0b4b[name=xxx]
    at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
    at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:628)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: An existing connection was forcibly closed by the remote host
    at sun.nio.ch.SocketDispatcher.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(Unknown Source)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
    at sun.nio.ch.IOUtil.read(Unknown Source)
    at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
    at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:136)
    at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:306)
    at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
    ... 6 more

slave logs:

<===[JENKINS REMOTING CAPACITY]===>Slave.jar version: 2.53.3
Slave.jar version: 2.53.3
This is a Windows slave
This is a Windows slave
Connection terminated
Connection terminated
ERROR: Failed to install restarter
hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
    at hudson.remoting.Request.abort(Request.java:297)
    at hudson.remoting.Channel.terminate(Channel.java:847)
    at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
    at hudson.remoting.Channel$1.handle(Channel.java:501)
    at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61)
    at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:594)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
    at ......remote call to slave-tester(Native Method)
    at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
    at hudson.remoting.Request.call(Request.java:172)
    at hudson.remoting.Channel.call(Channel.java:780)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.install(JnlpSlaveRestarterInstaller.java:52)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.access$000(JnlpSlaveRestarterInstaller.java:33)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$1.call(JnlpSlaveRestarterInstaller.java:39)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$1.call(JnlpSlaveRestarterInstaller.java:36)
    at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: hudson.remoting.Channel$OrderlyShutdown
    at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
    at hudson.remoting.Channel$1.handle(Channel.java:501)
    at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61)
    at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:594)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    ... 4 more
Caused by: Command close created at
    at hudson.remoting.Command.<init>(Command.java:56)
    at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1074)
    at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1072)
    at hudson.remoting.Channel.close(Channel.java:1156)
    at hudson.remoting.Channel.close(Channel.java:1138)
    at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1079)
    at hudson.remoting.Channel$1.handle(Channel.java:501)
    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)

What is causing this issue and how to solve it?

Answer 1

Similar issues

• https://issues.jenkins-ci.org/browse/JENKINS-27251
• https://issues.jenkins-ci.org/browse/JENKINS-6817
• https://issues.jenkins-ci.org/browse/JENKINS-28155
• https://issues.jenkins-ci.org/browse/JENKINS-28826
• https://issues.jenkins-ci.org/browse/JENKINS-22932
• https://bugreports.qt.io/browse/QTQAINFRA-928
• https://stackoverflow.com/questions/36743908/jenkins-slave-connection-aborted
• https://mailman.owncloud.org/pipermail/jenkins/2015-July/006091.html
• https://mailman.boum.org/pipermail/tails-dev/2014-December/007732.htm

Possible solution

Based on this source, part of the stack trace:

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

and experiencing the same issue in the office upgrading the jdk7 to the latest oracle-jdk8 could solve the issue.

If the issue persists:

1. Please indicate exactly when the issue occurs so it could be possible to reproduce the issue
2. Please add the Jenkins build log of the build that is causing the issue. Perhaps there is a specific task that is executed by the job that is causing the issue.
3. View the windows task manager while building the job
4. Please add the log of one of the successful builds and the log of one of the failed builds

Answer 2

I removed the file which solved my problem rm -rf /var/lib/jenkins/secrets/jenkins.slaves.JnlpSlaveAgentProtocol.secret

INFO: Both error and output logs will be printed to /home/ubuntu/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.13 This is a Unix agent WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by jenkins.slaves.StandardOutputSwapper$ChannelSwapper to constructor java.io.FileDescriptor(int) WARNING: Please consider reporting this to the maintainers of jenkins.slaves.StandardOutputSwapper$ChannelSwapper WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Evacuated stdout ERROR: Failed to load the key: jenkins.slaves.JnlpSlaveAgentProtocol.secret java.lang.Error: Failed to load the key: jenkins.slaves.JnlpSlaveAgentProtocol.secret