Friday, 30 March 2012

Application Packaging Structure

In a standard J2EE application, modules are packaged depending on their functionality. It can be EAR, WAR or JAR as well. Today, we will have a little discussion on each module and its structure.

A J2EE application consists of one or more J2EE modules and one J2EE application deployment descriptor. An application deployment descriptor contains a list of the applications's modules and information on how to customize the application. A J2EE application consists of one or more Java Archive (JAR) files along with zero or more Resource Archive (RAR) files packaged into an Enterprise ARchive (EAR) file with an .ear extension.

A J2EE module consists of one or more J2EE components for the same container type and one component deployment descriptor of that type. A component deployment descriptor contains declarative data to customize the components in the module. A J2EE module without an application deployment descriptor can be deployed as a stand-alone J2EE module. Types of J2EE modules are as follows:

  • Web Application Archive (WAR): A web application is a collection of servlets, HTML pages, classes, and other resources that can be bundled and deployed to several J2EE application servers. A WAR file can consist of the following items: servlets, JSPs, JSP tag libraries, utility classes, static pages, client-side applets, beans, bean classes, and deployment descriptors (web.xml and optionally sun-web.xml).
  • EJB JAR File: The EJB JAR file is the standard format for assembling enterprise beans. This file contains the bean classes (home, remote, local, and implementation), all of the utility classes, and the deployment descriptors (ejb-jar.xml and sun-ejb-jar.xml). If the EJB component is an entity bean with container managed persistence, a .dbschema file and a CMP mapping descriptor, sun-cmp-mapping.xml, must be included as well.
  • Application Client Container JAR File: An ACC client is a Sun Java System Application Server specific type of J2EE client. An ACC client supports the standard J2EE Application Client specifications, and in addition, supports direct access to the Sun Java System Application Server. Its deployment descriptors are application-client.xml and sun-application-client.xml.
  • Resource RAR File: RAR files apply to J2EE CA connectors. A connector module is like a device driver. It is a portable way of allowing EJB components to access a foreign enterprise system. Each Sun Java System Application Server connector has a J2EE XML file, ra.xml.

Image 1

Lets take a look at them one by one. --

JAR - Java ARchive

It is a file format based on the ZIP file format and similar to ZIP format this is also used for aggregating many files (many be of different types) into one aggregate file. This aggregate file will have (.jar) extension. Application development using Java uses JAR files for many useful purposes. One of the most common use is to package all the .class files, image files, and other files required by an Applet into a JAR file, so that the download of a single file will have all the components downloaded at the client machine. Otherwise, we would require those many HTTP Connections to individually download each of the components and this will of course be a very tedious and time-consuming effort. Another popular usage is to bundle all the .class files, and other required component files of a typical subsystem into a JAR file and include that JAR file into the CLASSPATH of another Java application which requires the services of that subsystem. The maintenance and deployment becomes very easier in this case.

This archive file format is platform-independent format which has been fully written in Java. This format is capable of handling audio and image files in addition to the .class files. It's an open standard and fully extendable. JAR file consist of a ZIP archive and an optional Manifest file, which contains package and extension related data. The Manifest file will have the name '
MANIFEST.MF'. This manifest file belongs to the optional directtory named 'META-INF'. This directory is used to store package and extension configuration data, security related data, versioning related data, services related data, etc.

EJB container hosts enterprise java beans based on EJB API designed to provide extended business functionality such as declarative transactions, declarative method level security and multi-protocol support - more of RPC style of distributed computing. EJB container required EJB module to be packaged in JAR file having ejb-jar.xml file in META-INF folder.


WAR - Web ARchive

As the name suggests this file format is used to package all the components of a Web Application. It may contain JARs, JSPs, Servlets, HTMLs, GIFs, etc. The purpose of this archive format is same as that of the JAR - to make the deployment, shipping, and in turn the maintenance process easier. This will have an XML file named
web.xml as the Deployment Descriptor. This Deployment Descriptor is used by the Web Container to deploy the web application correctly.

EAR - Enterprise ARchive

An enterprise application may be composed of several Web Applications and other independent JARs. This archive file format is used in those cases to bundle all the components of an enterprise application into a single file. Again the purpose is same - making deployment, shipping, and hence the maintenance easier. Since, an enterprise application may have several Web Applications as its components, hence an EAR file may contain WARs and JARs. An EAR also contains an XML-based Deployement Descriptor, which is used by the Application Server to deploy the enterprise application correctly.

Below is the model of a typical EAR.

Image 2

Thursday, 29 March 2012

How to determine the memory consumption by a java process

- If you are on Linux :


 you can try to grep the java process by:

====
[nikhil@nikhil bin]$ ps auxwww |grep java
nikhil    22412     1 18 10:43 pts/4    00:00:02 /home/nikhil/jdk1.6.0_31/bin/java -Djava.util.logging.config.file=/home/nikhil/Studies/apache-tomcat-7.0.26/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/home/nikhil/Studies/apache-tomcat-7.0.26/endorsed -classpath /home/nikhil/Studies/apache-tomcat-7.0.26/bin/bootstrap.jar:/home/nikhil/Studies/apache-tomcat-7.0.26/bin/tomcat-juli.jar -Dcatalina.base=/home/nikhil/Studies/apache-tomcat-7.0.26 -Dcatalina.home=/home/nikhil/Studies/apache-tomcat-7.0.26 -Djava.io.tmpdir=/home/nikhil/Studies/apache-tomcat-7.0.26/temp org.apache.catalina.startup.Bootstrap start
====


- Navigate to jdk bin dir or setup the the system classpath like below :

export CLASSPATH=/home/nikhil/jdk1.6.0_31/bin/:$CLASSPATH:.:

- execute jmap command :

for heap   jmap -heap <process id>, you will get output like below :

====
[nikhil@nikhil bin]$ jmap -heap 22412
Attaching to process ID 22412, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.6-b01

using thread-local object allocation.
Parallel GC with 4 thread(s)

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 2034237440 (1940.0MB)
   NewSize          = 1310720 (1.25MB)
   MaxNewSize       = 17592186044415 MB
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 85983232 (82.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 31850496 (30.375MB)
   used     = 28962800 (27.621078491210938MB)
   free     = 2887696 (2.7539215087890625MB)
   90.93359174061214% used
From Space:
   capacity = 5242880 (5.0MB)
   used     = 0 (0.0MB)
   free     = 5242880 (5.0MB)
   0.0% used
To Space:
   capacity = 5242880 (5.0MB)
   used     = 0 (0.0MB)
   free     = 5242880 (5.0MB)
   0.0% used
PS Old Generation
   capacity = 84738048 (80.8125MB)
   used     = 1802824 (1.7193069458007812MB)
   free     = 82935224 (79.09319305419922MB)
   2.1275259963505415% used
PS Perm Generation
   capacity = 21757952 (20.75MB)
   used     = 15853608 (15.119178771972656MB)
   free     = 5904344 (5.630821228027344MB)
   72.86351215408509% used
====

The above output shows the memory usage by java process, let's go little deep with the details:


Jmap is a utility that connects to the JVM on default jmx port i.e:1099 by default and may vary from vendor to vendor.

1.
==
using thread-local object allocation.
Parallel GC with 4 thread(s)
==

suggests that there are 4 GC threads running with parallel GC algoritham.

2.
==
Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 2034237440 (1940.0MB)
   NewSize          = 1310720 (1.25MB)
   MaxNewSize       = 17592186044415 MB
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 85983232 (82.0MB)
==

suggests the heap config and sizes of different chunk. To get details of these check my article JVM memory configuration.

3.
==
Heap Usage:
PS Young Generation
Eden Space:
   capacity = 31850496 (30.375MB)
   used     = 28962800 (27.621078491210938MB)
   free     = 2887696 (2.7539215087890625MB)
   90.93359174061214% used
From Space:
   capacity = 5242880 (5.0MB)
   used     = 0 (0.0MB)
   free     = 5242880 (5.0MB)
   0.0% used
To Space:
   capacity = 5242880 (5.0MB)
   used     = 0 (0.0MB)
   free     = 5242880 (5.0MB)
   0.0% used
PS Old Generation
   capacity = 84738048 (80.8125MB)
   used     = 1802824 (1.7193069458007812MB)
   free     = 82935224 (79.09319305419922MB)
   2.1275259963505415% used
PS Perm Generation
   capacity = 21757952 (20.75MB)
   used     = 15853608 (15.119178771972656MB)
   free     = 5904344 (5.630821228027344MB)
   72.86351215408509% used
==

Above suggests the memory usage by different chunks of the process heap.

Note : Process heap is process size not the java heap.


Now if you want to dump the info of the objects, classes and references residing over heap, you can take a heap dump by following :

jmap -J-d64 -heap:format=b <proces id>

above is for 64 bit jvms, if you have a 32 bit java, try below:

jmap -heap:format=b <process id>

here format=b suggests that ,it is in binary form. You can also produce an ASCII one, but most of the time it is of no use, because majority of heap dump analyzers support binary or hprof format.


OK, now I want to know about my classloader instances. here we go :

jmap -permstat <process id>

======
[nikhil@nikhil bin]$ jmap -permstat 22412
Attaching to process ID 22412, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.6-b01
7387 intern Strings occupying 666440 bytes.
finding class loader instances ..Finding object size using Printezis bits and skipping over...
done.
computing per loader stat ..done.
please wait.. computing liveness............................................liveness analysis may be inaccurate ...
class_loader classes bytes parent_loader alive? type

<bootstrap> 1424 8722704   null   live <internal>
0x00000000d9f63548 0 0 0x0000000086c001c0 live org/apache/catalina/loader/WebappClassLoader@0x000000008276adf8
0x00000000d9f1efa8 0 0 0x0000000086c001c0 live org/apache/catalina/loader/WebappClassLoader@0x000000008276adf8
0x0000000086c47000 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086d17fa0 0 0 0x0000000086c00230 dead java/util/ResourceBundle$RBClassLoader@0x00000000821f4cb0
0x00000000d9e16c48 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46e18 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46de0 1 3088 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9e16c80 1 3144 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46e50 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46da8 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9d8a470 0 0 0x0000000086c001c0 live org/apache/catalina/loader/WebappClassLoader@0x000000008276adf8
0x0000000086c00ad0 12 78648   null   live sun/misc/Launcher$ExtClassLoader@0x0000000081bd0df0
0x00000000d9e16bb0 1 3144 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9e16ab8 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46ec0 1 3088 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c47098 1 3088 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9e16be8 1 3088   null   dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c47140 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9e16948 17 100592 0x0000000086c001c0 live org/apache/catalina/loader/WebappClassLoader@0x000000008276adf8
0x00000000d9e16b28 1 3088   null   dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c47108 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9e16af0 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46e88 1 3128 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46d70 1 3088 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c470d0 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46f68 1 3128 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46fa0 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c47060 1 3128   null   dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c471b0 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9e16a80 1 3088 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c46ef8 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9f30118 0 0 0x0000000086c001c0 live org/apache/catalina/loader/WebappClassLoader@0x000000008276adf8
0x0000000086c47178 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x00000000d9e16a48 1 3088   null   dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c471e8 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c00230 97 931728 0x0000000086c00ad0 live sun/misc/Launcher$AppClassLoader@0x0000000081c3f970
0x0000000086c46f30 1 3120 0x0000000086c001c0 dead sun/reflect/DelegatingClassLoader@0x0000000081a675e8
0x0000000086c001c0 699 5813752 0x0000000086c00230 live org/apache/catalina/loader/StandardClassLoader@0x0000000081d63dd0

total = 39 2278 15737720     N/A     alive=9, dead=30     N/A
======


OK Baj, I just want to see the objects that are alive on my heap :

jmap -histo <process id>

I am not going to paste the output here, because it would be very large  :) :P


Note :  jconsole is also a powerful tool from which you can determine the jvm runtime details. See the page
jconsole




If you are on Windows : get the process id from Task manger that appears on screen after pressing ctrl+alt+delete, by default it doesn't show the pid, however you can custmize this from the view settings, rest steps would be the same.

Basic JDK tools and utilities


As an Application Admin/ Middleware Engineer. We ought to know basic functionalities of JDK tools. I tried to list few of them and would describe them in next blogs.


javac        The compiler for the Java programming language.
   
java          The launcher for Java applications.


javadoc    API documentation generator. 


apt           Annotation processing tool. See Annotation Processing Tool for  program annotation processing.
    
appletviewer    Run and debug applets without a web browser.    


jar    Create and manage Java Archive (JAR) files.


jdb    The Java Debugger.    


javah    C header and stub generator. Used to write native methods.
  
javap    Class file disassembler    


extcheck    Utility to detect Jar conflicts.    


keytool    Manage keystores and certificates.    


jarsigner    Generate and verify JAR signatures.    


policytool    GUI tool for managing policy files.    


native2ascii    Convert text to Unicode Latin-1.    


rmic    Generate stubs and skeletons for remote objects.    


rmiregistry    Remote object registry service.    


rmid    RMI activation system daemon.    


serialver    Return class serialVersionUID.    


tnameserv    Provides access to the naming service.


idlj    Generates .java files that map an OMG IDL interface and enable an application written in the Java programming language to use CORBA functionality.


orbd    Provides support for clients to transparently locate and invoke persistent objects on servers in the CORBA environment. ORBD is used instead of the Transient Naming Service, tnameserv. ORBD includes both a Transient Naming Service and a Persistent Naming Service. The orbd tool incorporates the functionality of a Server Manager, an Interoperable Naming Service, and a Bootstrap Name Server. When used in conjunction with the servertool, the Server Manager locates, registers, and activates a server when a client wants to access the server.


servertool    Provides ease-of-use interface for the application programmers to register, unregister, startup, and shutdown a server.


pack200    Transforms a JAR file into a compressed pack200 file using the Java gzip compressor. The compressed packed files are highly compressed JARs, which can be directly deployed, saving bandwidth and reducing download time.


unpack200    Transforms a packed file produced by pack200 into a JAR file.


htmlconverter    Converts an HTML page (file) containing applets to the OBJECT / EMBED tag format for Java Plug-in.


javaws    Command line tool for launching Java Web Start and setting various options.


jvisualvm    A graphical tool that provides detailed information about the Java technology-based applications (Java applications) while they are running in a Java Virtual Machine. Java VisualVM provides memory and CPU profiling, heap dump analysis, memory leak detection, access to MBeans, and garbage collection.See Java VisualVM for more information.


jconsole    A JMX-compliant graphical tool for monitoring a Java virtual machine. It can monitor both local and remote JVMs. It can also monitor and manage an application.


schemagen     Schema generator for Java Architecture for XML Binding.


wsgen     Tool to generate JAX-WS portable artifacts.


wsimport     Tool to generate JAX-WS portable artifacts.


xjc     Binding compiler for Java Ardchitecture for XML Binding.


jrunscript    Script shell for Java - Runs a script.


and the ones that admins love :)


jps     JVM Process Status Tool - Lists instrumented HotSpot Java virtual machines on a target system.


jstat     JVM Statistics Monitoring Tool - Attaches to an instrumented HotSpot Java virtual machine and collects and logs performance statistics as specified by the command line options.


jstatd     JVM jstat Daemon - Launches an RMI server application that monitors for the creation and termination of instrumented HotSpot Java virtual machines and provides a interface to allow remote monitoring tools to attach to Java virtual machines running on the local system.


jinfo    Configuration Info for Java - Prints configuration information for a given process or core file or a remote debug server.


jhat    Heap Dump Browser - Starts a web server on a heap dump file (eg, produced by jmap -dump), allowing the heap to be browsed.


jmap    Memory Map for Java - Prints shared object memory maps or heap memory details of a given process or core file or a remote debug server.


jsadebugd    Serviceability Agent Debug Daemon for Java - Attaches to a process or core file and acts as a debug server.


jstack    Stack Trace for Java - Prints a stack trace of threads for a given process or core file or remote debug server.


Certainly listing the tools only points to avail options, wait for upcoming blogs for detailed descriptions :)

Create new Admin user and password for weblogic co...


It happens with most of the admins that they install Weblogic application server and forget the username and password of the console .. There are few who think only editing boot.properties file is going to help them, but that's not true... so what to do next ?? Follow below steps and reset the username/password :

Remember to take back-up of all the files mentioned below.

Before starting the process go to

Note: DomainUsingWLST is the domain that we have created and now we want to reset the username and password. (If  we forgot them or for security reasons we want to change them.)

Please use .cmd or .sh according to your operating system.

Please go to:

WL_HOME\user_projects\domains\ DomainUsingWLST \security. 

 A file DefaultAuthenticatorInit.ldift is in this folder- as a precaution, save this file somewhere before proceeding/overwriting.

Step 1:

Open commands prompt and navigate to your weblogic Domain directory’s bin folder. Run setDomainEnv.cmd/sh or setWLSEnv

D:\bea10.3\user_projects\domains\DomainUsingWLST\bin\>setDomainEnv.cmd/sh


Step 2:

Enter:Java weblogic.security.utils.AdminAccount NewAdminUser NewAdminPassword . Where, NewAdminUser and NewAdminPassword are the User and Password you would like to create.  Don’t forget the trailing dot "." it tells the command that this should be run in the domain directory.

D:\bea10.3\user_projects\domains\DomainUsingWLST>
java weblogic.security.utils.AdminAccount weblogic1 weblogic1 ./

(Here weblogic1 weblogic1 are username and password respectively)

Step 3:

 The Above command creates a file, DefaultAuthenticatorInit.ldift, in the Domain directory.  This file needs to be moved to the directory

WL_HOME\user_projects\domains\ DomainUsingWLST \security. 

However, the old DefaultAuthenticatorInit.ldift is in this folder- as a precaution, save this old file somewhere before proceeding/overwriting.

Step 4:

Update the boot.properties  file, located in:

WL_HOME\user_projects\domains\ DomainUsingWLST \servers\AdminServer\security

Change the username and password to the username and password you created in step 2 (the values will be encoded, replace with clear text).


Step 5:

Rename ldap dir located at

WL_HOME\user_projects\domains\ DomainUsingWLST \servers\AdminServer\data


Step 6:

Return to the Domain Directory, and start the server.


Step 7:

Now go to admin console and try to open it using new username and password.


The admin user and password will be reset.


Note : WL_HOME = weblogic home