LDN - Luminis Developer Network

Today is our Winter registration, which is typically our heaviest load day of the year. I'll be keeping track of performance in this blog throughout the day.

Last year, we squeaked by with a v490 2xIV+ cpu for the oracle database, and v490 4xIV+ cpu for the ldap tier, and a T2000 running the web-front. Noting the load was far higher on the oracle db, we swapped the zones after registration ended.

So it will be interesting this year to see if performance increases with that change.

Last night, I removed two old terms, and ran cptool sync coursememberships and cptool synch data calendar. This morning, I shut down all the services, and reindexed the ldap. Hopefully that will increase our performance, removing a couple terms of oracle data.

We are still on patch 4.0.1.60, the same as last winter reg.

8:00am registration period went fine.

10:00am registration we are starting to see a lot higher database peaks, and have close to 1k concurrent sessions now. The ldap and web front are running fine.

10:06am 1284 sessions, 80% oracle cpu use, spikes from 15% to 40% on ldap, web server consistently under 20% cpu. Logins are very fast.

12:00am was similar to 10am.
1:40pm 9000 unique logins so far.
1:45pm took some screenshots of our java heap usage. I do not see any appreciable leaks (our webserver hangs about every 3 weeks of normal use, so something, suspected leak, is occuring, but appears to not be related to usage given the screenshots today).

2:00pm Hammer hit:( 2pm is our biggest block of students, at 9,000, plus everyone earlier that didn't register at the right time.

I am noticing what usually happens when a large amount of people try to enter at once, the database cpu peaks, followed by a sharp decline, and during that db decline, the web server cpu peaks. See-saws back and forth until the rush passes. see 'ouch.png' files.

Logins are taking a minute or so.