Using the same virtual server I tried to install Glassfish V3 (prelude) which comes in a 26MB zip-file. At least I would avoid the installer complaining about not enough memory available. After unzipping the server and with proper JAVA_HOME path all we need to do is to start the server with /glassfishv3-prelude/glassfish/bin/asadmin start-domain
Starting: java -jar -Xmx128m admin-cli.jar start-domain (here with 128MB setting)
I would get the same error in the logfile in domain1/logs/jvm.log
<tty> <writer thread='-1209660528'/> Error occurred during initialization of VM Could not reserve enough space for object heap <tty_done stamp='0.013'/> </tty> <hotspot_log_done stamp='0.013'/> </hotspot_log>
Conclusion (again): You need more than the 256MB priced virtual server.