There are 2 crucial phases you want to look under the hood of your running Glassfish or inside the JVM underneath: Performance Tuning and Health Monitoring during production.
With JMX (Java Management Extensions, Wikipedia) at hand, there are a few options to choose from.
The graphical monitoring tool is great for local deployment, it allows you to connect to a JVM on the same host or a remote host. It creates line graphs for your for all relevant from the moment you connect, it is perfect to observe a server while you do some testing or other actions, though it does not record any values while you are not connected. I have a hard time to get it running on a remote server and I do not favour the ‘open’ approach (see previous blog entry) which allows anyone to access the JVM with the disabled authentication settings. I also had situations where the JVM was frozen and it was no longer possible to access the JVM for monitoring, here I would rather have snapshots before the problem started together with server.log.
Glassfish Rest Interface
Note: You need to enable the areas you want to monitor with the admin console (or the asadmin command line) because per default all are OFF.
Then you access the monitoring values from the browser, curl or another application (authorization required!) through the URI, like https://localhost:4848/monitoring/domain/server/jvm/memory
This also very useful to explore the available values for the asadmin approach (next option)
You can access all these telemetry stuff also through the asadmin command line ($GLASSFISH_HOME/bin), this comes handy for administrator who want to run cronjobs or bash scripts to take snapshots.
./asadmin –user myadminuser –passwordfile /opt/passwordfile.txt get –monitor server.jvm.thread-system.*
Note: you need to pass admin user and password, the passwordfile contains the password in clear text:
glassfish@ip-10-10-10-10:/opt/app-server/bin# ./asadmin --user admin --passwordfile ~/pwd.txt get --monitor server.jvm.memory.* server.jvm.memory.committedheapsize-count-count = 3914924032 server.jvm.memory.committedheapsize-count-description = Amount of memory in bytes that is committed for the Java virtual machine to use server.jvm.memory.committedheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.committedheapsize-count-name = CommittedHeapSize server.jvm.memory.committedheapsize-count-starttime = 1364197189330 server.jvm.memory.committedheapsize-count-unit = bytes server.jvm.memory.committednonheapsize-count-count = 259194880 server.jvm.memory.committednonheapsize-count-description = Amount of memory in bytes that is committed for the Java virtual machine to use server.jvm.memory.committednonheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.committednonheapsize-count-name = CommittedNonHeapSize server.jvm.memory.committednonheapsize-count-starttime = 1364197189330 server.jvm.memory.committednonheapsize-count-unit = bytes server.jvm.memory.dotted-name = server.jvm.memory server.jvm.memory.initheapsize-count-count = 4244635648 server.jvm.memory.initheapsize-count-description = Amount of memory in bytes that the Java virtual machine initially requests from the operating system for memory management server.jvm.memory.initheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.initheapsize-count-name = InitialHeapSize server.jvm.memory.initheapsize-count-starttime = 1364197189330 server.jvm.memory.initheapsize-count-unit = bytes server.jvm.memory.initnonheapsize-count-count = 24313856 server.jvm.memory.initnonheapsize-count-description = Amount of memory in bytes that the Java virtual machine initially requests from the operating system for memory management server.jvm.memory.initnonheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.initnonheapsize-count-name = InitialNonHeapSize server.jvm.memory.initnonheapsize-count-starttime = 1364197189330 server.jvm.memory.initnonheapsize-count-unit = bytes server.jvm.memory.maxheapsize-count-count = 3914924032 server.jvm.memory.maxheapsize-count-description = Maximum amount of memory in bytes that can be used for memory management server.jvm.memory.maxheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.maxheapsize-count-name = MaxHeapSize server.jvm.memory.maxheapsize-count-starttime = 1364197189330 server.jvm.memory.maxheapsize-count-unit = bytes server.jvm.memory.maxnonheapsize-count-count = 587202560 server.jvm.memory.maxnonheapsize-count-description = Maximum amount of memory in bytes that can be used for memory management server.jvm.memory.maxnonheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.maxnonheapsize-count-name = MaxNonHeapSize server.jvm.memory.maxnonheapsize-count-starttime = 1364197189330 server.jvm.memory.maxnonheapsize-count-unit = bytes server.jvm.memory.objectpendingfinalizationcount-count-count = 0 server.jvm.memory.objectpendingfinalizationcount-count-description = Approximate number of objects for which finalization is pending server.jvm.memory.objectpendingfinalizationcount-count-lastsampletime = 1364268183797 server.jvm.memory.objectpendingfinalizationcount-count-name = ObjectsPendingFinalization server.jvm.memory.objectpendingfinalizationcount-count-starttime = 1364197189330 server.jvm.memory.objectpendingfinalizationcount-count-unit = count server.jvm.memory.usedheapsize-count-count = 1841024784 server.jvm.memory.usedheapsize-count-description = Amount of used memory in bytes server.jvm.memory.usedheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.usedheapsize-count-name = UsedHeapSize server.jvm.memory.usedheapsize-count-starttime = 1364197189330 server.jvm.memory.usedheapsize-count-unit = bytes server.jvm.memory.usednonheapsize-count-count = 236054792 server.jvm.memory.usednonheapsize-count-description = Amount of used memory in bytes server.jvm.memory.usednonheapsize-count-lastsampletime = 1364268183797 server.jvm.memory.usednonheapsize-count-name = UsedNonHeapSize server.jvm.memory.usednonheapsize-count-starttime = 1364197189330 server.jvm.memory.usednonheapsize-count-unit = bytes Command get executed successfully.
You can use wildcards, but be aware, server.* can create easily 50.000 + lines, even server.jvm.* 5.000 + lines.
Only one parameter can be passed.
An option is to snapshot this by a cronjob into a datetime stamped file every 1 or 5 minutes.
- Implement your own monitoring as part of your deployment, make sure your persist into a database or logfiles, so the information is accessible even the JVM is down.
Pro: You can implement to whatever extend you want.
Con: Learning curve for JMX.
- Use an existing solution that can be deployed into the application server
The only application I could find is LightFish by Adam Bien