Another registration day, Luminis IV hardware notes

5
Average: 5 (3 votes)

Having been through 2 registration days (winter and spring) with Luminis IV, I thought I'd share what I've learned about the hardware requirements for the system.

On our winter registration in November, we entered that day with the following hardware:

Resource/Ldap/Message Broker: Sun V490, 4x sparc IV+ cpus, 16 gigs of ram.
Oracle Database: Sun V245, 2x sparc IIIi, 8 gigs of ram.
Web Server: T2000, 1x 8 core 1.0ghz cpu, 8 gigs of ram
Identical second web server was not online for PD deployment due to load balancer problem.
Calendar: Sun V245, 2x sparc IIIi, 8 gigs of ram.

We expected anywhere from 1200 to 1800 sessions sustained, for about 30 minutes, in blocks at 8am, 10am, 12pm, and 2pm.

The day was horribly slow. This was Luminis 4.0.1.38 Both our database and web server were at 100% cpu usage for most of the day, with the web server taking as long as 2 minutes to log someone in.

Our servers were sized on the only real world data available at the time of their purchase, which was other schools using Luminis III in production. For instance, Georgia I believe, had 10 web front ends, each supporting around 2000 sustained sessions. Given that, we expected each of our front ends to handle ~1500-2000 sessions. And our database size was largely based on our previous Luminis experience, which was that the web server was always maxed on registration days, but the database was hardly touched.

So, before going into spring registration we did several things:
1. Added additional indexes to our oracle database. Our dba was able to identify a couple queries that were using the majority of the system resources. Upon looking at the tables, we could see that the query was selecting on fields that were not indexed. And those tables had hundreds of thousands of records. This happened each login.
2. patched to 4.0.1.60. This was recommended by Luminis support due to several caching improvements. This patch overwrote the personDirectory.xml file and the web.xml file, so beware if you have any custom entries in those files.
3. Since we installed all our servers in solaris 10 zones, we moved our production database and production web server to new hardware with the zfs send/receive command. We had just gotten new test hardware, a v490 and a T2000. Since we got the test hardware much later than production (long story:), it happened to be faster.

So our system going into spring registration:
Resource/Ldap/Message Broker: Sun V490, 4x sparc IV+ cpus, 16 gigs of ram.
Oracle Database: MOVED to V490, 2x sparc IV+ cpus, 16 gigs of ram.
Web Server: MOVED to T2000, 1x 8 core 1.2ghz cpu, 32 gigs of ram (appears to only use a max of 12 gigs of ram at peak use).
Identical second web server was not online for PD deployment due to load balancer problem.
Calendar: Sun V245, 2x sparc IIIi, 8 gigs of ram.

Our result: very nice registration. The web server barely crested 30% cpu, more often hovering around 15% cpu use. The oracle database sometimes hit 70% cpu, but was between 30% and 50% most of the day. During both registration periods, our resource/ldap/message broker server was barely touched.

So in the end, if I had this all over to do again, I'd build a system closer to this design for supporting 2400-4800 sessions:

Oracle database: most worked. V490, minimum of 2 sparc cpus, 4 would be much better for room to grow. Min of 12 gigs of ram. 16 recommended if you want to allot more to oracle than is defaulted.

2x Web Servers: T2000 1x 8 core 1.2ghz cpu. Min 12gigs of ram.

Resource/ldap/message broker: hardly seemed to work at all. I would not buy 4 processors for this system. And I may just swap our oracle zone and our resource zone when I get a chance.

Calendar: is fine at a v245 IIIi 8gigs with our use.
Email: currently is a v490 with 2 IV+ cpus. It is generally alright, although on certain days, it could use 4x IV+ cpus. 16gigs of ram seems fine.

So I hope that helps someone. I suspect that as higher patch states of Luminis are released, the hardware requirements go down drastically. But for any of you moving from Luminis III to Luminis IV, be very aware that the database seems to be a huge bottleneck in Luminis as of patch 60.

I hope that Sungard updates their hardware req. guides at some point in the future, because the following downloaded today from their support site isn't anywhere near being able to support 4000 users.

From the Luminis LP40001reqs.pdf on the connect.sungardhe.com site.
Three Server Configuration for Solaris 9 or 10
0-4,000
Luminis Platform server + Calendar: Sun Fire v245, two UltraSparc III 1.5 GHz or T2000: UltraSparc TI 4 Core 1.0 GHz processors, 4 GB RAM, appropriate hard drive space (as derived from the section “Disk space requirements” on page 12).

Mail server: Sun Fire v245, two UltraSparc III 1.5 GHz or T2000 UltraSparc TI 4 Core 1.0 GHz processors, 2 GB RAM, appropriate hard drive space (as derived from the section “Disk space requirements” on page 12).

Database server
Contact your database vendor to determine the impact of the Luminis Platform on the size or processing power of any existing database servers or for details on sizing a new database server that will be dedicated to the Luminis Platform components.
^^^^
Always been curious why SCT/Sungard thinks that Sun or Oracle would have any idea how large to size a database server for their product heh. Although, we did have a meeting with some Sun reps recently. Sungard and Sun have teamed up and created a testing lab to help accurately size hardware in multiple configurations. So hopefully by the time that Luminis 5+ arrives, we'll have more accurate data to go on.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Similar registration experience

Even though we're a small college (1250 FTE), we had a disasterous registration both for Fall and for Spring. By disasterous, I mean we had to stop registration, restart the system, and limit the number of concurrent registrants to about 120 just to limp through it. It has become a hot button issue with senior staff and a recurring topic at faculty meetings.

We've had Sungard on campus reviewing our system in December and remotely load testing our system this month to determine why our performance has been so dismal, though we meet or exceed the published spec. They haven't been able to give me an answer yet, but considering your experience and mine, I think I'm going to migrate to a 8 core 16 or 32GB memory T2 server before we register in April, if for no other reason than to keep my job.

May I ask what Oracle tables you indexed?

New attachments

I just emailed you the sql scripts that helped us decide which indexes to add, as well as the actual indexes we added. (also attached them to this post for anyone else interested)

A couple notes: make sure you get patched up to at least 4.0.1.60, make sure you set your concurrent logins way way down on the registration day (ours was set at 3 concurrent logins). The most intensive part of luminis is logging in.

Also like I mentioned above, the web server only used 12 gigs of the 32 gigs of ram in it. This would be dependent upon your java heap settings, among other things. I increased our heap from default, to give the web server more memory to use. In the tomcat-cp-conf file in cp_root/bin our heap looks like this:
CATALINA_OPTS=$CATALINA_OPTS$JDD_SEP"-Xmx1833125376"
CATALINA_OPTS=$CATALINA_OPTS$JDD_SEP"-Xmx1833125376"
CATALINA_OPTS=$CATALINA_OPTS$JDD_SEP"-XX:MaxPermSize=128m"