Home » forums » Luminis Platform / System

Can I avoid server restarts when Jdbc Data source in personDirectory.xml goes bad

Submitted by libdms on Mon, 2009-11-09 10:38

Approximately once  a fortnight, one of our database connections (JDBC) used in personDirectory.xml goes bad. The only way we know of to retrieve data again is to restart Luminis.

From the time it goes bad, until the restart, the cp.log on the affected Portal webtier fills up with entries like

[2009-11-06 12:20:55,978] [ERROR](ParallelTimeoutMergingPersonAttributeDaoImpl.java:413) {http-80-Processor75} [com.pipeline.uportal.persondir.ParallelTimeoutMergingPersonAttributeDaoImpl]: JdbcPersonAttrViaPdsLoginIdDaoImpl.getUserAttributes({username=361948607983385}) failed to complete with exception
java.util.concurrent.ExecutionException: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [ SELECT someAttribs FROM UolTable WHERE username='AUsername'

Sungard UDC also say that there is no way to regain DB connectivity for personDirectory.xml except for a webserver restart

 

Does anyone have a way of recovering from this without a restart? I have started looking at the uPortal documentation to try and understand how it works. So far it is going slowly.

I am guessing that the answer lies in

1) How the JdbcPersonAttributeDaoImpl connects to the Remote Db

2) Whether the same connection is shared between users or connections are opened and closed

3) If a custom piece of Java can be written to close the Db connection / or refresh the Db pool

I can supply a lot more information if that will help anyone.

Derek

University of Leeds, UK

 

 

update from Sungard (cannot add as a comment at the moment)

Hi Derek,

From looking at the code it seems it uses the uportalpooled jdbc connection pool.

‹ Luminis issues with IE 8 Querying Luminis Database ›

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

sure the answer is out there somewhere

Submitted by libdms on Fri, 2009-11-13 09:55.

Can anyone remind me how to get Luminis to reconnect to its own database (possible for one of the two schemas) without a webserver restart?

Derek
University of Leeds, UK

faq 1-3mktdw

Submitted by libdms on Fri, 2009-11-13 13:13.

sungard have pointed me at a configman setting

com.campuspipeline.rdb.conn.ConnectionPool.getPooledConnection.validateconnection

This allows the default DB connection to uPortal to have a more graceful recovery, but I am not sure whether this can be applied to my remote database. One of the biggest obstacles is thinking of a way of testing this without actually impacting on University wide services.

Derek

University of Leeds, UK

re jdbc connection goes bad

Submitted by Jason on Fri, 2009-11-13 15:29.

 Is it 'going bad' because it goes for a long period without activity?  I know we used to see the banner to luminis connectivity break if banner were down for maintenance for a long time.  It would require a luminis restart.

If so, I wonder if you could cron/script something to query the database from within the luminis framework every so often?

very unlikely that it is quiet

Submitted by libdms on Mon, 2009-11-16 06:36.

We are seeing the 'going bad' at very busy times of the day (and possibly at quiet times too)

The personDirectory.xml is called every time a user logs in to our portal.

We have seen it 'going bad' when we have 2500 concurrent logins (shared between two web servers - until we take one off-line to resolve the DB connectivity)

It may be that when there are a lot of user sessions starting - i.e. the login form is triggered - that the external DB struggles to respond to all of the requests quickly enough. Luminis will then timeout/give up/throw the teddy bear out of the pram (we do live in a fast food society). Once Luminis has decided to sulk it is too late!

If I get time today, I hope to experiment on a dev server.

1) add in more parameters to the personDirectory.xml DB connection - especially "validateconnection", but also some timeouts etc.

2) point it at a dev db that I can stop and start ad nauseum

3) see if I can get Luminis into the 'gone bad' state (with current parameters)

4) see if I can get Luminis to see the DB after a DB restart

Will report back.

With regard to the cron, I was wondering whether it would be possible to write something to get the DB connection from the pool and close it, reopen it (hopefully this would also put it into a 'good state') (it must be Monday if I am being this optimistic)

 

Derek

University of Leeds, UK

bad connection

Submitted by Jason on Mon, 2009-11-16 15:31.

http://www.ja-sig.org/wiki/display/UPC/JdbcPersonAttributeDaoImpl

Odd, I've never experienced that type of problem.  I suppose you can try increasing the property values.

 

		<property name="removeAbandoned">
			<value>true</value>
		</property>
		<property name="maxActive">
			<value>5</value>
		</property>
		<property name="maxIdle">
			<value>3</value>
		</property>
		<property name="maxWait">
			<value>10000</value>
		</property>

 

Since it's not a idle issue, scripting a call to the db wouldn't help. I tried googling for info on JdbcPersonAttributeDaoImpl and other related pieces...talk about sparse documentation.  There must be a way to write a little java class to manipulate/call/close/open the connection, but I couldn't find it.  
 

Syndicate content