Home › Forums › Luminis Platform / System

cptool problem

Tue, 2008-04-29 09:33 — riversace

I'm running Lum IV on Linux and have developed a cptool problem. We're not quite sure when the problem started or why but here it is ..

When using get user, set user and import ims, ( there are probably others), I get the following message...

ERROR: Event notification subsystem initialization failed. This is
> most likely because the JMS provider is not accepting connections.
> Aborting command.

I haven't changed anything, and those with poking rights say they haven't either...it just stopped working.

I've opened a sr with sungard on the problem about a month ago, but their response has been slow.

I've been poking around blindly on my own and have found a few things...

mbtool list brokerstatus says everything is fine, but I can't access the web interface to the broker.

I've stopped and restarted luminis many times. I find this message in cpstart log...

Starting messagebroker (Mon Apr 28 12:45:31 EDT 2008)
>>> -----------------------------------------------------
>>> messagebroker ...... starting
>>> messagebroker ...... started
>>> campuspipeline.messaging.admin.MessageBrokerAdminException: web
>>> services start validation timeout
>>> at
>>> campuspipeline.messaging.admin.MessageBrokerAdmin.waitForState(MessageBrokerAdmin.java:1867)
>>> at
>>> campuspipeline.messaging.admin.MessageBrokerAdmin.startWebServices(MessageBrokerAdmin.java:1970)
>>> at
>>> campuspipeline.messaging.admin.MessageBrokerAdmin.startBroker(MessageBrokerAdmin.java:289)
>>> at
>>> campuspipeline.messaging.admin.MbTool.processCommand(MbTool.java:395)
>>> at
>>> campuspipeline.messaging.admin.MbTool.execute(MbTool.java:322)
>>> at campuspipeline.messaging.admin.MbTool.main(MbTool.java:273)

I see a bunch of these messages in /opt/luminis/products/tomcat/tomcat-mb/logs/catalina.out ...

463646 [http-7677-Processor25] ERROR org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/mb].[authen] - Allocate exception for servlet authen
com.sct.pipeline.configuration.ConfigurationException: Unable to find configbootstrap.properties file for configuration.

The only configbootstrap.properties file is here and has the same permissions as the one on our production system...

/opt/luminis/config/configbootstrap.properties

All of the env variables match the production system as well.

So there is what I know.

Any insight as to what I can do or where I can look for problems would be greatly appreciated.

Rick Versace.
CIS Vassar College
riversace@vassar.edu
845-437-5915

0
No votes yet
‹ Luminis IV Parallel Deployment Questions Learning JMeter ›

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Tue, 2008-04-29 14:18 — Jason

cptool problem

Ours does act the same way from time to time. There are a few things to do, but basically what you need to do is 'reboot' (the general fix for everything:)

Before you do though, there are a few things to check:
1. make sure that all your broker connections are active. You can either go into the web admin tool or from the command line, like:
mbtool list destinations
then from that list, check them all. I wrote a script to do this.
mbtool list substatus -dest=(one value at a time from the mbool list destinations list)

They should all be Durable Sub. State: ACTIVE (assuming you use them all, I'm not sure of your particular needs).

If one is in a disabled state, restarting the resource tier/ldap webserver can change it to active, sometimes. Othertimes, you need to make sure that your configman settings didn't get changed to false. No clue why it happens, but I've seen luminis turn off my connections sometimes. Like
-bash-3.00$ configman -g messaging.datapipeline.* | grep -i enabled
messaging.datapipeline.client.cp$broadcastcontrol.enabled=true
messaging.datapipeline.client.cp$broadcastresponse.enabled=true
messaging.datapipeline.client.cp$calendar.enabled=true
messaging.datapipeline.client.cp$cpentityevent.enabled=false
messaging.datapipeline.client.cp$cpincoming.enabled=true
messaging.datapipeline.client.cp$hotbackup.enabled=true
messaging.datapipeline.client.cp.enabled=true
messaging.datapipeline.client.ldi$logger.enabled=true
messaging.datapipeline.client.mb$logger.enabled=true

I'll look through some of my old tickets about this and see if I can find some other tricks.

*edit: I've had 11 tickets on this (the older ones seem to have disappeared from my helpdesk ticket archive) in the last couple years or so. There isn't any sure way of fixing it. Most of those 11 tickets, I respond with, "Well, it seems to be working again" hehe.

It could be something blatant though, so make sure to turn on the debug mode for your broker and lmg.
: You can debug the event flow (through LMG to LMB) as follows:

1) In $CP_WEBINF/config/cplog4j.properties:

log4j.logger.datapipeline.audit=DEBUG, datapipeline-audit

2) In $SCT_LMG_HOME/Events/config/logging.properties:

log4j.rootCategory=DEBUG
log4j.category.com=DEBUG, out
log4j.category.com.sctcorp.events.MessageAdapter=DEBUG

Once these are set, restart LMG and capture the following logs after reproducing the issue. When reproducing the error, please note the time stamp or any other identifying information about the event in question, such as the eventID # or the event type.

$SCT_LMG_HOME/Events/logs/adapter.log
$CP_ROOT/logs/datapipeline-audit.log

-----

3) To debug the broker itself, run "configman -g messaging.jms.provider.configfile" to find the IMQ configuration file location. Edit this file, and change imq.log.level from INFO to DEBUG. Save the file, and the additional log verbosity will be recorded in $CP_ROOT/logs/mb-broker.log.