Fisheye as a service - Won't launch

[eKyNoX]

Hello,

According the FishEye wrapper installation process page : https://confluence.atlassian.com/display/FISHEYE/Running+FishEye+as+a+Windows+service#RunningFishEyeasaWindowsservice-environvariables

I can't launch FishEye as a service using wrapper.

To give you max informations, I used jdk7 from Oracle download in x32 version as suggested by the tutorial. Also, I filled out all the informations as needed.

Here is the config file : http://paste2.org/p/2674683

And the output log error :

x4 times before ...
------------------------------------------------------------
ERROR  | wrapper  | 2012/12/31 14:05:09 | JVM exited while loading the application.
STATUS | wrapper  | 2012/12/31 14:05:14 | Launching a JVM...
INFO   | jvm 5    | 2012/12/31 14:05:14 | java version "1.7.0_10"
INFO   | jvm 5    | 2012/12/31 14:05:14 | Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
INFO   | jvm 5    | 2012/12/31 14:05:14 | Java HotSpot(TM) Server VM (build 23.6-b04, mixed mode)
INFO   | jvm 5    | 2012/12/31 14:05:14 | 
INFO   | jvm 5    | 2012/12/31 14:05:14 | Syntaxe : java [-options] class [args...]
INFO   | jvm 5    | 2012/12/31 14:05:14 |            (pour l'exécution d'une classe)
INFO   | jvm 5    | 2012/12/31 14:05:14 |    ou  java [-options] -jar jarfile [args...]
INFO   | jvm 5    | 2012/12/31 14:05:14 |            (pour l'exécution d'un fichier JAR)
INFO   | jvm 5    | 2012/12/31 14:05:14 | où les options comprennent :
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -d32	  utilisez le modèle de données 32 bits s'il est disponible
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -d64	  utilisez le modèle de données 64 bits s'il est disponible
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -client	  pour sélectionner la machine virtuelle "client"
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -server	  pour sélectionner la machine virtuelle "server"
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -hotspot	  est un synonyme pour la machine virtuelle "client"  [en phase d'abandon]
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   La machine virtuelle par défaut est client.
INFO   | jvm 5    | 2012/12/31 14:05:14 | 
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -cp <class search path of directories and zip/jar files>
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -classpath <class search path of directories and zip/jar files>
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   Liste de répertoires, d'archives JAR et
INFO   | jvm 5    | 2012/12/31 14:05:14 |                    d'archives ZIP séparés par des ;, dans laquelle rechercher les fichiers de classe.
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -D<name>=<value>
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   définition d'une propriété système
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -verbose[:class|gc|jni]
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   activation de la sortie en mode verbose
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -version      impression de la version du produit et fin de l'opération
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -version:<value>
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   exécution de la version spécifiée obligatoire
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -showversion  impression de la version du produit et poursuite de l'opération
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -jre-restrict-search | -no-jre-restrict-search
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   inclusion/exclusion des environnements JRE privés de l'utilisateur dans la recherche de version
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -? -help      impression du message d'aide
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -X            impression de l'aide sur les options non standard
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -ea[:<packagename>...|:<classname>]
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -enableassertions[:<packagename>...|:<classname>]
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   activation des assertions avec la granularité spécifiée
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -da[:<packagename>...|:<classname>]
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -disableassertions[:<packagename>...|:<classname>]
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   désactivation des assertions avec la granularité spécifiée
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -esa | -enablesystemassertions
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   activation des assertions système
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -dsa | -disablesystemassertions
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   désactivation des assertions système
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -agentlib:<libname>[=<options>]
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   chargement de la bibliothèque d'agent natif <libname>, par exemple -agentlib:hprof
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   voir également, -agentlib:jdwp=help et -agentlib:hprof=help
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -agentpath:<pathname>[=<options>]
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   chargement de la bibliothèque d'agent natif via le chemin d'accès complet
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -javaagent:<jarpath>[=<options>]
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   chargement de l'agent du langage de programmation Java, voir java.lang.instrument
INFO   | jvm 5    | 2012/12/31 14:05:14 |     -splash:<imagepath>
INFO   | jvm 5    | 2012/12/31 14:05:14 |                   affichage de l'écran d'accueil avec l'image spécifiée
INFO   | jvm 5    | 2012/12/31 14:05:14 | Voir http://www.oracle.com/technetwork/java/javase/documentation/index.html pour plus de détails.
ERROR  | wrapper  | 2012/12/31 14:05:14 | JVM exited while loading the application.
FATAL  | wrapper  | 2012/12/31 14:05:14 | There were 5 failed launches in a row, each lasting less than 300 seconds.  Giving up.
FATAL  | wrapper  | 2012/12/31 14:05:14 |   There may be a configuration problem: please check the logs.
STATUS | wrapper  | 2012/12/31 14:05:14 | <-- Wrapper Stopped

Well, it found the jdk but the command to start FishEye doesn't looks like good.

Any idea ?

Thanks.

[parthiban subramaniam · Accepted solution]

can imagine your frustration. Things which are supposed to be 5min work can sometime become things that get you spend useless amount of time scratching your head..

Apologies i was not aware of this issue with wrapper in windows.

Lets start fresh.

Configure your fisheye to use wrapper but use 32bit JVM

Use only 256mb of permsize

wrapper.java.additional.12=-XX:MaxPermSize=256m

modify these parameters to use 1024m

wrapper.java.initmemory=1024

wrapper.java.maxmemory=1024

these changes should be safe parameters for your 32bit jvm, is your installation fresh or has many repositories?

lets hope we get lucky this time.

[eKyNoX]

Well, removed all java installations, desinstalled the service. Deleted FishEye (& application folder) & Wrapper Folders.

Now ready to re-start installation.

Uncompress FishEye Installation folder => put the FishEye folder on C:\Program Files (x64)\Atlassian\FishEye... => reinstalled java jdk-6u38-windows-i586 => redo system environment variables (FISHEYE_INST with folder creation at the right place, and JAVA_HOME) => copied config.xml to FishEye instance directory => uncompressed wrapper directory and files and copied to FishEye home directory => edited wrapper.conf => installed service => launched it (no errors) ...

And that works ! I really don't know why there was a problem...

[parthiban subramaniam]

are you able to start this up and get it running with out wrapper involved?

are you logged in as the same user who is trying to launch fisheye?

to rule out corrupt instlation can you, do a fresh install some where else and try it out?

[eKyNoX]

Well, the start from windows prompt is ok, the user is the same but from the page :

https://confluence.atlassian.com/display/FISHEYE/Running+FishEye+as+a+Windows+service

That tells :

Warning when using 64-bit Java JDK

When using a 64-bit Java JDK with the wrapper obtained via the link in the install instructions above, you may see the following in the wrapper.log file:

WARNING - Unable to load the Wrapper's native library 'wrapper.dll'. The file is located on the path at the following location but could not be loaded: C:\installs\service\fisheye28\wrapper\lib\wrapper.dll.

Please verify that the file is readable by the current user and that the file has not been corrupted in any way. System signals will not be handled correctly.

This is caused by using a 64-bit JDK (even on a 64-bit machine). Changing to a 32-bit version of the JDK will prevent this warning. Community Edition versions of the 64-bit Windows Java Service Wrapper are not currently available.

I can't use other versions of java cause it cause memory errors, and can't use the 64 bits version either. Fresh install done, no changes, so the wrapper do not work. Now what to do ? launch with the console normally seems to be the only way. The "FishEyE install as a service" page can be removed or refactored as well.

It's the snake that bites its own tail.

[parthiban subramaniam]

Windows is always crazy when it comes to memory managment and there is not much of a clear documentation around.

I hope you have set your pagefile to be the same size as the physical memory available on your machine?

If not set it to that and try going up to 2048mb if this fails install 64bit JDK/JVM and utilize that.

Hope this helps

cheers

[eKyNoX]

I hope you have set your pagefile to be the same size as the physical memory available on your machine?

Yes.

Tried with the other jdk6_x64, works with max 2048 now, no more memory error but vm appears hung, like the log show :

http://paste2.org/p/2694266

Sometimes, it says it can't load wrapper.dll, file is present and can be opened normally with that user. Sometimes it crash, generally it appears hung. No access to the localhost adress...

[parthiban subramaniam]

-XX:MaxPermSize=2048m

should be

wrapper.java.additional.12=-XX:MaxPermSize=512m

dont go beyond that infact try 256m and if it fails increase it to 512m on the above parameter, if it still fails then its not able to resevere and is being ignored

the other two parameters should be

wrapper.java.initmemory=1024

wrapper.java.maxmemory=2048

Is there anything in the system that prevents an user from reserving these memory?

[eKyNoX]

Hello again and merry good year 2013 (we still alive yeah!)

So, I tried again with different parameters :

wrapper.java.additional.12=-XX:MaxPermSize=512m ------------> OK
wrapper.java.initmemory=1024 --------------> OK

wrapper.java.maxmemory=2048 -------------> NOT GOOD

I can't go upper than 1600, or it don't launch.

ps: still have memory errors with that parameters so.

Is there anything in the system that prevents an user from reserving these memory?

I admit this idea passed in my head, I went to system advanced properties to re-activate swap share windows file (hibersys or pagesys, do not remember which one it is) and then restart the server, no changes. I do not think anything else prevent for service (wrapper.exe I suppose) to use the memory it needs, but I could be wrong.

[parthiban subramaniam]

you are still running out of memory,

Try increasing these parameters and ensure you have enough memory in your system to cope with fishey demand.

wrapper.java.additional.12=-XX:MaxPermSize=512

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=2048

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=2048

cheers

[eKyNoX]

STATUS | wrapper | 2012/12/31 18:36:25 | Launching a JVM...
INFO | jvm 2 | 2012/12/31 18:36:25 | Error occurred during initialization of VM
INFO | jvm 2 | 2012/12/31 18:36:25 | Could not reserve enough space for object heap
INFO | jvm 2 | 2012/12/31 18:36:25 | Could not create the Java virtual machine.
ERROR | wrapper | 2012/12/31 18:36:25 | JVM exited while loading the application.

There's almost 55% of 8GB DDRAM free

[eKyNoX]

There's free memory space, np.

When I modify memory parameters :

1. wrapper.java.additional.12=-XX:MaxPermSize=2048m

2. wrapper.java.initmemory=1024

3. wrapper.java.maxmemory=1600

Well, if I try to increase wrapper.java.maxmemory (for example 2048), the service do not want to start. I tried comment the line wrapper.java.additional.12.... I got the same results.

So I tried with many different values, or I got OutOfMemory error, or it do not want to start... starting to pulling hair.

[parthiban subramaniam]

INFO | jvm 1 | 2012/12/31 15:15:27 | java.lang.OutOfMemoryError: PermGen space

that is from your erro log, you are running out of memory, modify your memory 

settings and increase the permgen space

wrapper.java.additional.12=-XX:MaxPermSize=128m -- should be 256mb 

like so 

wrapper.java.additional.12=-XX:MaxPermSize=256m

i'm out of this game for a while, please make sure jdk 1.7 is supported by fisheye if not use the 

latest and greatest of 1.6.XX version 

cheers

[eKyNoX]

I tested with jdk6 with this first log pastebin : http://paste2.org/p/2675098 as an answer to Rodrigo.

I tested again with 256m replacement, here is the result : http://paste2.org/p/2675125

Still can't access to local adress.

[Rodrigo Girardi Adami]

Hi,

Somehow it seems this would be a failure in the JVM itself as for this part:

JVM exited while loading the application.

FATAL | wrapper | 2012/12/31 14:05:14 | There were 5 failed launches in a row, each lasting less than 300 seconds. Giving up.

FATAL | wrapper | 2012/12/31 14:05:14 | There may be a configuration problem: please check the logs.

What I would advise is to try to use another JDK version like the latest version of 1.6.0 for example and try to run the service again. If this doesn't help, then please open a ticket for Atlassian support so we can investigate that.

Hope that helps!

Regards,

Rodrigo

[eKyNoX]

I added a new comment with a new pastebin, the error is not the same as before and the service launch but just can't access the adress.

[eKyNoX]

I checked with jdk6, http://localhost:8060 still do not show.

Here is the results :http://paste2.org/p/2675098

I remind that it work with manual command line launch.

[parthiban subramaniam]

Check your JVM configuration flags, there might be unsupported flags in use. or valid JVM flags with a space of invalid charachter inbetween.

your log file is bit muddled up, it will be helpful if you could clear out the logs, do a start up and attach the files here

cheers

[eKyNoX]

Hello and thanks for answer.

I found the bad line in config, it was :

wrapper.java.additional.4=-Dcom.sun.management.jmxremote

So I changed it by :

#wrapper.java.additional.4=-Dcom.sun.management.jmxremote

and the service launched correctly!

======================================================================

Now I have another problem,

I can't connect to my FishEye on http://localhost:8060 when started with wrapper as a service.

Here is the error I get :

http://paste2.org/p/2674901

I tried to launch it manually with cmd.exe, it works correctly.