Unable to register remote RHQ-Agent "The agent will now wait until it has registered with the server"
8:04 a.m.
Hi,
I am unable to register remote rhq-agent (3.0 final) with the server (3.0
final).
I am able to ping the remote RHQ-Agent from the RHQ-Server and RHQ-Server
from the Remote RHQ-Agent. Also I am able to connect / telnet into
RHQ-Server's tcp:7080 from the client and RHQ-Agent's (the system has NAT,
but can connect to the publicly advertised IP as found in
http://myipaddress.com) tcp:16163 from the RHQ-Server. The client does not
have any active firewall and all the requisite ports tcp:16163, tcp:7080,
tcp:7443 are open on the server side.
I have gone through the FAQ and the blog entry
http://pilhuhn.blogspot.com/2008/08/rhq-tip-of-day-agent-waiting-at-start...
rhq-agent.bat --clean produces
Agent Name [Mobilapps] : TNSB
Agent Hostname or IP Address [!*] : 172.16.0.252
Agent Port [16163] :
RHQ Server Hostname or IP Address [172.16.0.252] : 122.181.162.143
RHQ Server Port [7080] :
The setup has been completed for the preferences at node
[/rhq-agent/default].
The agent will now wait until it has registered with the server...
The server has rejected the agent registration request. Cause:
[org.rhq.core.clientapi.server.core.AgentRegistrationException:Server cannot
ping the agent's endpoint. The agent's endpoint is probably invalid or there
is a firewall preventing the server from connecting to the agent. Endpoint:
socket://
172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&numAcceptThreads=1&maxPoolSize=303&clientMaxPoolSize=304&socketTimeout=60000&enableTcpNoDelay=true&backlog=200
]
Will retry the agent registration request soon...
the netstat on the remote RHQ-Agent produces
TCP 172.16.0.252:5900 ABTS-KK-Static-ILP-130:4787 ESTABLISHED
TCP 172.16.0.252:16163 ABTS-KK-Static-ILP-130:3165 ESTABLISHED
TCP 172.16.0.252:55630 ABTS-KK-Static-ILP-143:7080 ESTABLISHED
TCP 172.16.0.252:55631 ABTS-KK-Static-ILP-143:7080 TIME_WAIT
TCP 172.16.0.252:55633 ABTS-KK-Static-ILP-143:7080 TIME_WAIT
Why the agent is unable to register / throwing errors?
--
Thanks & Regards
Prashant N
8:22 a.m.
New subject: Unable to register remote RHQ-Agent "The agent will now wait until it has registered with the server"
Hi,
Read the yellow note that begins "If the agent fails to register with
the server" on
http://www.rhq-project.org/display/JOPR2/RHQ+Agent+Installation#RHQAgentI....
Go to Administration > High Availability > Servers in the GUI, and make
sure the Server address (or addresses) specified there are all addresses
that are resolvable and reachable from the Agent machine.
- Ian
On 08/11/2010 09:04 AM, Prashant N wrote:
Hi,
I am unable to register remote rhq-agent (3.0 final) with the server
(3.0 final).
I am able to ping the remote RHQ-Agent from the RHQ-Server and
RHQ-Server from the Remote RHQ-Agent. Also I am able to connect /
telnet into RHQ-Server's tcp:7080 from the client and RHQ-Agent's (the
system has NAT, but can connect to the publicly advertised IP as found
in http://myipaddress.com) tcp:16163 from the RHQ-Server. The client
does not have any active firewall and all the requisite ports
tcp:16163, tcp:7080, tcp:7443 are open on the server side.
I have gone through the FAQ and the blog entry
http://pilhuhn.blogspot.com/2008/08/rhq-tip-of-day-agent-waiting-at-start...
rhq-agent.bat --clean produces
Agent Name [Mobilapps] : TNSB
Agent Hostname or IP Address [!*] : 172.16.0.252
Agent Port [16163] :
RHQ Server Hostname or IP Address [172.16.0.252] : 122.181.162.143
RHQ Server Port [7080] :
The setup has been completed for the preferences at node
[/rhq-agent/default].
The agent will now wait until it has registered with the server...
The server has rejected the agent registration request. Cause:
[org.rhq.core.clientapi.server.core.AgentRegistrationException:Server
cannot ping the agent's endpoint. The agent's endpoint is probably
invalid or there is a firewall preventing the server from connecting
to the agent. Endpoint:
socket://172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&numAcceptThreads=1&maxPoolSize=303&clientMaxPoolSize=304&socketTimeout=60000&enableTcpNoDelay=true&backlog=200
<http://172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&...>]
Will retry the agent registration request soon...
the netstat on the remote RHQ-Agent produces
TCP 172.16.0.252:5900 <http://172.16.0.252:5900>
ABTS-KK-Static-ILP-130:4787 ESTABLISHED
TCP 172.16.0.252:16163 <http://172.16.0.252:16163>
ABTS-KK-Static-ILP-130:3165 ESTABLISHED
TCP 172.16.0.252:55630 <http://172.16.0.252:55630>
ABTS-KK-Static-ILP-143:7080 ESTABLISHED
TCP 172.16.0.252:55631 <http://172.16.0.252:55631>
ABTS-KK-Static-ILP-143:7080 TIME_WAIT
TCP 172.16.0.252:55633 <http://172.16.0.252:55633>
ABTS-KK-Static-ILP-143:7080 TIME_WAIT
Why the agent is unable to register / throwing errors?
--
Thanks & Regards
Prashant N
_______________________________________________
rhq-users mailing list
rhq-users(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-users
--
Ian Springer
Sr. Software Developer
JBoss Operations Network
Red Hat
ian.springer(a)redhat.com
8:51 a.m.
New subject: Unable to register remote RHQ-Agent "The agent will now wait until it has registered with the server"
all the requisite ports tcp:16163, tcp:7080, tcp:7443 are open on the
server side.
for the record, the server-side doesn't need 16163 - that's the (only)
port the agent listens on. So your firewall needs to allow 16163 traffic
to from FROM the server TO the agent box.
The server has rejected the agent registration request. Cause:
[org.rhq.core.clientapi.server.core.AgentRegistrationException:Server
cannot ping the agent's endpoint. The agent's endpoint is probably
invalid or there is a firewall preventing the server from connecting to
the agent. Endpoint:
When the server receives a registration request from an agent, before
the server grants the request it will try to ping the agent on the
endpoint the agent is registering. If this check fails, the registration
fails and the agent gets this error.
In almost all cases, its due to one of two things:
1) the agent's IP is not resolvable or reachable from the server box
2) there is a firewall preventing the server from reaching the agent
over that IP and port.
7:36 a.m.
New subject: Unable to register remote RHQ-Agent "The agent will now wait until it has registered with the server"
On Wed, Aug 11, 2010 at 7:21 PM, John Mazzitelli <mazz(a)redhat.com> wrote:
> all the requisite ports tcp:16163, tcp:7080, tcp:7443 are open
on the
server side.
for the record, the server-side doesn't need 16163 - that's the (only)
port the agent listens on. So your firewall needs to allow 16163 traffic
to from FROM the server TO the agent box.
> The server has rejected the agent registration request. Cause:
> [org.rhq.core.clientapi.server.core.AgentRegistrationException:Server
> cannot ping the agent's endpoint. The agent's endpoint is probably
> invalid or there is a firewall preventing the server from connecting to
> the agent. Endpoint:
When the server receives a registration request from an agent, before
the server grants the request it will try to ping the agent on the
endpoint the agent is registering. If this check fails, the registration
fails and the agent gets this error.
In almost all cases, its due to one of two things:
1) the agent's IP is not resolvable or reachable from the server box
2) there is a firewall preventing the server from reaching the agent
over that IP and port.
_______________________________________________
rhq-users mailing list
rhq-users(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-users
As stated in my problem statement, I am able to reach the agent from the
server and vice-versa through the ping command. Also when the agent is
active / started I can telnet into the agent's IP Address:16163. Hence the I
am ruling out the possibility of firewall in between as there are no
firewalls involved on the client side.
WRT HA / Server config: The End Point Address is the public IP
(122.181.162.143) of the server and is reachable from outside.
What else could be the problem ... How should I solve this ?
--
Thanks & Regards
Prashant N
8:51 a.m.
As stated in my problem statement, I am able to reach the agent from
the
server and vice-versa through the ping command. Also when the agent is
active / started I can telnet into the agent's IP Address:16163. Hence
the I am ruling out the possibility of firewall in between as there are
no firewalls involved on the client side.
WRT HA / Server config: The End Point Address is the public IP
(122.181.162.143) of the server and is reachable from outside.
What else could be the problem ... How should I solve this ?
I have never seen this failure when it wasn't one of those two things I
mentioned earlier. Therefore, the only thing I can suggest is that you
look in the server logs for any types of error messages the server might
be logging that we missed.
In addition, you can add this to the
<rhq-server-install-dir>/jbossas/server/default/conf/jboss-log4j.xml:
<category
name="org.rhq.enterprise.server.core.comm.ServerCommunicationsService">
<priority value="DEBUG"/>
</category>
Wait for that change to get hot deployed (server polls for changes every
60 seconds, so it won't take long). Then restart your agent. Look for
this to be logged at the DEBUG level and tell us what you see:
"Failed to ping endpoint [<your agent endpoint>]. Cause: <find out what
this exception message says>"
3:46 a.m.
Sorry for the delayed reply. After enabling the debug output on the server,
here is what I get in the server logs
--------------------------------------------------------------------------------------------------------------------
2010-08-13 17:37:47,690 ERROR
[org.rhq.enterprise.communications.command.client.ClientCommandSenderTask]
{ClientCommandSenderTask.send-failed}Failed to send command [Command:
type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.timeout=10000,
rhq.send-throttle=true}];
params=[{targetInterfaceName=org.rhq.enterprise.communications.Ping,
invocation=NameBasedInvocation[ping]}]]. Cause:
java.util.concurrent.TimeoutException:null. Cause:
java.util.concurrent.TimeoutException
2010-08-13 17:37:47,791 DEBUG
[org.rhq.enterprise.server.core.comm.ServerCommunicationsService]
{ServerCommunicationsService.ping-failed}Failed to ping endpoint [socket://
172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&numAcceptThreads=1&maxPoolSize=303&clientMaxPoolSize=304&socketTimeout=60000&enableTcpNoDelay=true&backlog=200].
Cause: java.lang.reflect.UndeclaredThrowableException:null ->
java.util.concurrent.TimeoutException:null
2010-08-13 17:38:36,736 WARN
[org.rhq.enterprise.communications.command.server.CommandProcessor]
{CommandProcessor.failed-authentication}Command failed to be authenticated!
This command will be ignored and not processed: Command: type=[remotepojo];
cmd-in-response=[false]; config=[{rhq.agent-name=tnsb,
rhq.externalizable-strategy=AGENT, rhq.send-throttle=true}];
params=[{targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService,
invocation=NameBasedInvocation[getLatestPlugins]}]
2010-08-13 17:39:07,809 ERROR
[org.rhq.enterprise.communications.command.client.ClientCommandSenderTask]
{ClientCommandSenderTask.send-failed}Failed to send command [Command:
type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.timeout=20000,
rhq.send-throttle=true}];
params=[{targetInterfaceName=org.rhq.enterprise.communications.Ping,
invocation=NameBasedInvocation[ping]}]]. Cause:
java.util.concurrent.TimeoutException:null. Cause:
java.util.concurrent.TimeoutException
2010-08-13 17:39:07,809 DEBUG
[org.rhq.enterprise.server.core.comm.ServerCommunicationsService]
{ServerCommunicationsService.ping-failed}Failed to ping endpoint [socket://
172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&numAcceptThreads=1&maxPoolSize=303&clientMaxPoolSize=304&socketTimeout=60000&enableTcpNoDelay=true&backlog=200].
Cause: java.lang.reflect.UndeclaredThrowableException:null ->
java.util.concurrent.TimeoutException:null
2010-08-13 17:40:48,020 ERROR
[org.rhq.enterprise.communications.command.client.ClientCommandSenderTask]
{ClientCommandSenderTask.send-failed}Failed to send command [Command:
type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.timeout=10000,
rhq.send-throttle=true}];
params=[{targetInterfaceName=org.rhq.enterprise.communications.Ping,
invocation=NameBasedInvocation[ping]}]]. Cause:
java.util.concurrent.TimeoutException:null. Cause:
java.util.concurrent.TimeoutException
2010-08-13 17:40:48,020 DEBUG
[org.rhq.enterprise.server.core.comm.ServerCommunicationsService]
{ServerCommunicationsService.ping-failed}Failed to ping endpoint [socket://
172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&numAcceptThreads=1&maxPoolSize=303&clientMaxPoolSize=304&socketTimeout=60000&enableTcpNoDelay=true&backlog=200].
Cause: java.lang.reflect.UndeclaredThrowableException:null ->
java.util.concurrent.TimeoutException:null
2010-08-13 17:42:08,039 ERROR
[org.rhq.enterprise.communications.command.client.ClientCommandSenderTask]
{ClientCommandSenderTask.send-failed}Failed to send command [Command:
type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.timeout=20000,
rhq.send-throttle=true}];
params=[{targetInterfaceName=org.rhq.enterprise.communications.Ping,
invocation=NameBasedInvocation[ping]}]]. Cause:
java.util.concurrent.TimeoutException:null. Cause:
java.util.concurrent.TimeoutException
2010-08-13 17:42:08,040 DEBUG
[org.rhq.enterprise.server.core.comm.ServerCommunicationsService]
{ServerCommunicationsService.ping-failed}Failed to ping endpoint [socket://
172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&numAcceptThreads=1&maxPoolSize=303&clientMaxPoolSize=304&socketTimeout=60000&enableTcpNoDelay=true&backlog=200].
Cause: java.lang.reflect.UndeclaredThrowableException:null ->
java.util.concurrent.TimeoutException:null
2010-08-13 17:43:48,663 ERROR
[org.rhq.enterprise.communications.command.client.ClientCommandSenderTask]
{ClientCommandSenderTask.send-failed}Failed to send command [Command:
type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.timeout=10000,
rhq.send-throttle=true}];
params=[{targetInterfaceName=org.rhq.enterprise.communications.Ping,
invocation=NameBasedInvocation[ping]}]]. Cause:
java.util.concurrent.TimeoutException:null. Cause:
java.util.concurrent.TimeoutException
2010-08-13 17:43:48,663 DEBUG
[org.rhq.enterprise.server.core.comm.ServerCommunicationsService]
{ServerCommunicationsService.ping-failed}Failed to ping endpoint [socket://
172.16.0.252:16163/?rhq.communications.connector.rhqtype=agent&numAcceptThreads=1&maxPoolSize=303&clientMaxPoolSize=304&socketTimeout=60000&enableTcpNoDelay=true&backlog=200].
Cause: java.lang.reflect.UndeclaredThrowableException:null ->
java.util.concurrent.TimeoutException:null
--------------------------------------------------------------------------------------------------------------------
Here the client has a static IP Address on 172.16.0.252 and connects to the
internet via DSL Cable which provides static public IP. This case would be
true for the agents / clients which would be behind corporate firewalls to
have static DMZ IP and then Firewall would have a static Public IP - aka
NAT.
On the client side, I am binding the service to its IP, 172.16.0.252 and
agent / server complain that this is IP is not pingable. I cannot bind the
agent to public IP as it throws error
How to solve this?
On Thu, Aug 12, 2010 at 7:21 PM, John Mazzitelli <mazz(a)redhat.com> wrote:
> As stated in my problem statement, I am able to reach the agent
from the
> server and vice-versa through the ping command. Also when the agent is
> active / started I can telnet into the agent's IP Address:16163. Hence
> the I am ruling out the possibility of firewall in between as there are
> no firewalls involved on the client side.
>
> WRT HA / Server config: The End Point Address is the public IP
> (122.181.162.143) of the server and is reachable from outside.
>
> What else could be the problem ... How should I solve this ?
I have never seen this failure when it wasn't one of those two things I
mentioned earlier. Therefore, the only thing I can suggest is that you
look in the server logs for any types of error messages the server might
be logging that we missed.
In addition, you can add this to the
<rhq-server-install-dir>/jbossas/server/default/conf/jboss-log4j.xml:
<category
name="org.rhq.enterprise.server.core.comm.ServerCommunicationsService">
<priority value="DEBUG"/>
</category>
Wait for that change to get hot deployed (server polls for changes every
60 seconds, so it won't take long). Then restart your agent. Look for
this to be logged at the DEBUG level and tell us what you see:
"Failed to ping endpoint [<your agent endpoint>]. Cause: <find out what
this exception message says>"
_______________________________________________
rhq-users mailing list
rhq-users(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-users
--
Thanks & Regards
Prashant N
5022
days inactive
5028
days old
rhq-users@lists.fedorahosted.org
5 comments
3 participants
participants (3)
-
Ian Springer -
John Mazzitelli -
Prashant Nagaraja